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December  6,  2007 


MEMORANDUM  FOR  DIRECTOR,  DEFENSE  CONTRACT  MANAGEMENT  ' 

AGENCY 

AUDITOR  GENERAL,  DEPARTMENT  OF  THE  ARMY 


SUBJECT :  Report  on  the  Acquisition  of  Surface-Launched  Advanced  Medium-Range 
Air-to-Air  Missile  (Report  No.  D-2008-032) 


We  are  providing  this  report  for  review  and  comment.  We  considered 
management  comments  on  a  draft  of  this  report  when  preparing  the  final  report.  This 
report  addresses  the  Army’s  management  of  the  Surface-Launched  Advanced  Medium- 
Range  Air-to-Air  Missile  Program. 

DoD  Directive  7650.3  requires  that  all  recommendations  be  resolved  promptly. 
The  Acting  Director,  Defense  Contract  Management  Agency  nonconcurred  with 
Recommendation  C.2.  Additionally,  the  Army  Chief  Information  Officer  did  not  respond 
to  Recommendation  D.l.  and  the  Product  Manager,  Surface-Launched  Advanced 
Medium-Range  Air-to-Air  Missile  did  not  respond  to  Recommendation  D.2.  Therefore 
we  request  that  the  Acting  Director,  Defense  Contract  Management  Agency  provide 
additional  comments  on  Recommendation  C.2.  and  that  the  Army  Chief  Information 
Officer  and  the  Product  Manager,  Surface-Launched  Advanced  Medium-Range  Air-to- 
Air  Missile  provide  comments  on  Recommendations  D.l.  and  D.2.  respectively  by 
January  8,  2008. 

If  possible,  please  send  management  comments  in  electronic  format  (Adobe 
Acrobat  file  only)  to  AUDACM@dodig.mil.  Copies  of  the  management  comments  must 
contain  the  actual  signature  of  the  authorizing  official.  We  cannot  accept  the  /  Signed  / 
symbol  in  place  of  the  actual  signature.  If  you  arrange  to  send  classified  comments 
electronically,  they  must  be  sent  over  the  SECRET  Internet  Protocol  Router  Network 
(SIPRNET). 

We  appreciate  the  courtesies  extended  to  the  staff.  Questions  should  be  directed 
to  Mr.  Harold  C.  James  at  (703)  604-9088  (DSN  664-9088).  See  Appendix  D  for  the 
report  distribution.  The  team  members  are  listed  inside  the  back  cover. 

By  direction  of  the  Deputy  Inspector  General  ipr  Auditing: 


Richard  B.  Jolliffe 
Assistant  Inspector  General 
Acquisition  and  Contract  Management 


Special  Warning 

This  report  contains  information  that  ip  designated  »FOR  OFFICIAL  USE  ONLY;”  in  naaordansa 
’TTTttr  tht  PitLiluin  ullului nnrtjpn  Ail  under  nemptiwna  3,  4,  and  G.  Section  title  IS,  United 

ninnrthw facet  diailuam  t  of  company  confidential  oi*  proprietary  informntioni  Thie  report  mutt  ha 
safeguarded  in  accordance  with  BoD  Regulation  S.I00.7.B. 


FOR  OFFICIAL  USE  QNL¥ 


Department  of  Defense  Office  of  Inspector  General 


Report  No.  D-2008-032 

(Project  No.  D2007-D000AE-0060.000) 


December  6,  2007 


Acquisition  of  the  Surface-Launched  Advanced 
Medium-Range  Air-to-Air  Missile 


Executive  Summary 


Who  Should  Read  This  Report  and  Why?  Defense  officials  responsible  for 
generating,  processing,  and  approving  acquisition  requirements  and  managing  weapon 
system  acquisitions  should  read  this  report.  This  report  discusses  defining  capability 
requirements,  planning  the  systems  engineering  effort,  defining  responsibilities  of  the 
Defense  Contract  Management  Agency  (DCMA),  and  establishing  information  assurance 
requirements  in  support  of  the  low-rate  initial  production  decision  for  the  Surface- 
Launched  Advanced  Medium-Range  Air-to-Air  Missile  (SLAMRAAM). 


Background.  The  SLAMRAAM  is  an  air  defense  system  the  Army  is  developing  to 
protect  U.S.  and  friendly  forces  from  aerial  surveillance  and  air  attack  by  cruise  missiles, 
unmanned  aerial  vehicles,  unmanned  combat  aerial  vehicles,  and  rotary-  and  fixed-wing 
aircraft.  The  SLAMRAAM  will  provide  the  Army  the  means  to  prevent  enemies  from 
penetrating  friendly  forces,  while  freeing  commanders  to  synchronize  mo^menLmd 
firenower.  The  Army  will  emnlov  the  SLAMRAAM  in  units  that  are^^^^^^^™ 


is  ot  Juty  2UU  /,  tne  program  s  running  to  devetop  ana  procure  the 
system  totaiea  $02^5  million,  with  $208.3  million  in  research,  development,  test,  and 
evaluation  funds  and  $414.2  million  for  procurement  of  hardware  (including  69  fire  units, 
35  integrated  fire  control  stations,  55  Advanced  Medium-Range  Air-to-Air  Missiles,  and 
30  sensor  kits  to  be  installed  on  the  Sentinel  Radar). 


Results.  We  detennined  that  the  Army  could  have  more  cost  effectively  prepared  the 
program  for  the  low-rate  initial  production  decision  as  evidenced  by  the  need  for  the 
SLAMRAAM  Program  Office  to  rebaseline  the  development  contract  because  of 
contractor  technical  difficulties  and  program  funding  shortfalls  that  resulted  in  increased 
contract  costs.  Specific  areas  needing  management  attention  are  discussed  below. 

The  Director  of  Combat  Developments,  U.S.  Army  Air  Defense  Artillery  School  had  not 
yet  sufficiently  defined  capability  requirements  for  the  SLAMRAAM  in  the  draft 
capability  production  document  being  prepared  to  support  the  low-rate  initial  production 
decision  planned  for  FY  2010.  Until  the  Director  of  Combat  Developments  adheres  to 
policy  for  defining  system  capability  requirements,  the  Anny  will  be  hindered  in  its 
ability  to  ensure  that  the  SLAMRAAM  satisfies  essential  warfighter  capability 
requirements  (finding  A). 

The  Product  Manager  for  SLAMRAAM  began  developing  a  systems  engineering  plan 
but  did  not  use  it  to  support  its  systems  engineering  management  approach  for  the 
SLAMRAAM.  Instead,  the  product  manager  was  using  the  contractor-developed 
systems  engineering  management  plan.  By  not  using  a  systems  engineering  plan  that 
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included  entrance  and  success  criteria  for  planned  technical  reviews  and  defined  the 
applied  systems  engineering  processes,  the  product  manager  did  not  avail  the  program 
office  with  a  system  engineering  management  approach  needed  to  cost  effectively 
manage,  develop,  and  integrate  the  system  to  fully  meet  program  requirements 
(finding  B). 

The  Commander,  DCMA  Space  Sensors  and  Communications  Operations,  Raytheon 
Integrated  Defense  Systems,  and  the  Project  Manager,  Cruise  Missile  Defense  System 
established  a  memorandum  of  agreement  for  the  SLAMRAAM  program  that  did  not 
reference  current  DCMA  policy  and  fully  establish  links  between  the  memorandum  of 
agreement  performance  metrics  and  the  project  manager’s  desired  outcomes. 
Additionally,  the  commander  did  not  fully  describe  DCMA  activities  to  support  the 
SLAMRAAM  program  in  the  surveillance  plan  nor  establish  letters  of  delegation  with 
other  DCMA  contract  management  offices  for  surveillance  of  all  SLAMRAAM 
subcontractors.  Until  the  commander  and  the  project  manager  implement  those  actions, 
the  commander  will  not  be  able  to  provide  the  project  manager  with  the  most  informed 
recommendations  regarding  contractor  progress  towards  attaining  contract  cost,  schedule, 
and  performance  requirements  (finding  C). 

The  SLAMRAAM  Product  Office  changed  to  an  information  assurance  strategy  that  does 
not  comply  with  issued  and  approved  DoD  information  assurance  requirements. 
Specifically,  the  product  office,  as  directed,  was  following  the  DoD  Information 
Assurance  Certification  and  Accreditation  Program  guidance  that  was  in  coordination 
within  DoD  but  not  approved  for  implementation  in  DoD.  The  DoD  Information 
Technology  Security  Certification  and  Accreditation  Process  remains  the  official  DoD 
information  assurance  guidance.  Until  the  DoD  Information  Assurance  Certification  and 
Accreditation  Program  guidance  is  properly  coordinated  and  approved,  the  product  office 
places  the  information  contained  in  the  SLAMRAAM  system  at  greater  risk  of  loss, 
misuse,  unauthorized  access,  or  modification  (finding  D). 

The  problems  described  in  our  findings  resulted  from  material  internal  control 
weaknesses  in  the  management  of  the  SLAMRAAM.  The  Background  section  discusses 
these  material  internal  control  weaknesses. 

Management  Comments  and  Audit  Response.  We  received  comments  from  the 
Acting  Director,  Defense  Contract  Management  Command;  the  Army  Project  Manager, 
Cruise  Missile  Defense  System;  and  the  Army  Director  of  Combat  Developments,  U.S. 
Anny  Air  Defense  Artillery  School.  We  did  not  receive  comments  from  the  Anny  Chief 
Infonnation  Officer. 

The  Acting  Director,  Defense  Contract  Management  Agency  and  the  Project  Manager, 
Cruise  Missile  Defense  System  concurred  with,  or  proposed  actions  meeting  the  intent  of, 
recommendations  for  updating  the  memorandum  of  agreement  between  their 
organizations  to  reference  current  policy  and  guidance  and  to  establish  an  annex  linking 
performance  metrics  and  standards  to  the  customer-desired  outcome  for  the 
SLAMRAAM.  The  Acting  Director  nonconcurred  with  making  surveillance  plans  or 
activity  annexes  a  mandatory  part  of  the  memorandums  of  agreement  with  acquisition 
program  managers.  He  stated  that  the  Defense  Contract  Management  Agency  would 
instead  use  strategies  contained  in  the  individual  perfonnance  plans  of  functional 
specialists  to  ensure  execution  of  activities  to  achieve  performance  commitments.  The 
Anny  Project  Manager,  Cruise  Missile  Defense  System  concurred  with  revising  the  draft 
systems  engineering  plan  to  include  entrance  and  success  criteria  for  planned  technical 
reviews  and  a  full  description  of  the  systems  engineering  processes  applied  to  the 
SLAMRAAM.  The  project  manager  did  not  provide  comments  on  a  recommendation  to 
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return  to  following  the  DoD  Information  Technology  Security  Certification  and 
Accreditation  Process  in  developing  the  SLAMRAAM.  Though  not  required  to 
comment,  the  project  manager  concurred  with  revising  the  draft  capability  production 
document  for  the  SLAMRAAM  to  establish  measurable  and  testable  capability 
requirements  but  nonconcurred  with  defining  the  * 

*  as  a  key  performance  parameter.  The  Anny 

Director  of  Combat  Developments,  U.S.  Anny  Air  Defense  Artillery  School  concurred 
with  revising  the  draft  capability  production  document  to  define  * 

*  and 


to  establish  measurable  and  testable  capability  requirements. 


In  response  to  the  Acting  Director,  Defense  Contract  Management  Agency  comments,  we 
provided  additional  infonnation  concerning  the  advantages  of  including  surveillance 
plans  or  activities  annexes  in  memorandums  of  agreement  with  acquisition  program 
managers.  See  the  Findings  Section  of  the  report  for  a  discussion  of  management 
comments  on  the  recommendations  and  Appendix  C  for  a  discussion  of  management 
comments  on  the  report. 


Accordingly,  we  request  that  the  Acting  Director,  Defense  Contract  Management 
Agency;  the  Army  Chief  Information  Officer;  and  the  Product  Manager,  Surface- 
Launched  Advanced  Medium-Range  Air-to-Air  Missile  comment  on  the  final  report  by 
January  8,  2008. 


Freedom  of  Information  Act  Exemption  Number  5,  which  includes  internal  advice,  recommendations,  and 
subjective  evaluations,  as  stated  in  Army  Regulation  25-55,  “The  Department  of  the  Army  Freedom  of 
Information  Act  Program,”  November  1,  1997. 
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Background 


The  Army  Surface-Launched  Advanced  Medium-Range  Air-to-Air  Missile 
(SLAMRAAM)  is  a  major  system  that  is  in  the  system  development  and 
demonstration  phase  of  the  acquisition  process;  a  low-rate  initial  production 
(LRIP)  decision  is  planned  for  FY  2010. 

Mission  and  System  Description.  The  SLAMRAAM  is  an  air  defense  system 
that  the  Army  is  developing  to  protect  U.S.  and  friendly  forces  from  aerial 
surveillance  and  air  attack  by  cruise  missiles,  unmanned  aerial  vehicles, 
unmanned  combat  aerial  vehicles,  and  rotary-  and  fixed-wing  aircraft.  The 
SLAMRAAM  will  provide  the  Army  with  the  means  to  prevent  enemies  from 
penetrating  friendly  forces,  while  freeing  commanders  to  synchronize  movement 
and  firepower.  The  Anny  will  employ  the  SLAMRAAM  in  units  that  are 
integrated  in  a  netted  and  distributed  architecture  linked  to  other  Anny, 
interagency,  joint,  and  multinational  forces. 

The  SLAMRAAM  includes  four  subsystems:  the  fire  unit,  the  integrated  fire 
control  station  (IFCS),  the  Advanced  Medium-Range  Air-to-Air  Missile 
(AMRAAM),  and  the  Sentinel  Radar.  The  Product  Manager  for  SLAMRAAM  is 
developing  the  fire  unit  and  the  IFCS  and  integrating  the  AMRAAM  and  the 
Sentinel  Radar  into  the  SLAMRAAM  as  Government-furnished  equipment. 

•  Fire  Unit.  The  fire  unit,  a  highly  mobile  ancUransportable  launcher, 
will  consist  of  a  ready-to-fire  missile  load  of^H  AIM- 120  series 
AMRAAMs  mounted  on  a  high-mobility  multipurpose  wheeled 
vehicle.  The  fire  unit  will  be  manned  by  two  combat-equipped 
soldiers. 

•  IFCS.  The  IFCS,  which  will  perfonn  battle  management  functions, 
consists  of  a  shelter,  with  two  workstations,  mounted  on  a  high- 
mobility  multipurpose  wheeled  vehicle. 

•  AMRAAM.  The  SLAMRAAM  will  fire  AIM-120  series  AMRAAM 
missiles  that  are  currently  used  in  air-to-air  combat  engagements.  The 
AMRAAM  is  a  supersonic,  guided  missile  that  uses  active  radar  target 
acquisition,  proportional  navigation  guidance,  and  active  target 
tracking. 

• 


The  Army  plans  to  incrementally  replace  Stinger-based  air  defense  systems  in 
inventory  with  SLAMRAAMs  to  provide  an  increased  battlespace  defense  against 
cruise  missiles,  unmanned  aerial  vehicles,  fixed-wing,  and  rotary-wing  aircraft. 

Program  Management.  The  Product  Manager  for  SLAMRAAM  is  developing 
the  SLAMRAAM  for  the  Project  Manager,  Cruise  Missile  Defense  Systems 
(CMDS).  The  project  manager  is  responsible  for  equipping  the  transformation  of 


1 

FOR  OFFICIAL  USE  ONLY- 


b(3) 


the  existing  maneuver  air  and  missile  defense  capability  into  an  integrated  air  and 
missile  defense  system  of  systems.  In  addition  to  SLAMRAAM,  the  following 
CMDS  programs  will  be  part  of  this  system  of  systems:  Joint  Land  Attack  Cruise 
Missile  Defense  Elevated  Netted  Sensor  System,  Stinger-Based  Avenger  and 
Man-Portable  Air  Defense  System,  and  directed  energy  applications.  The  Anny 
Acquisition  Executive  delegated  the  milestone  decision  authority  responsibility 
for  the  SLAMRAAM  Program  to  the  Program  Executive  Officer,  Air  and  Space 
Missile  Defense,1  on  September  16,  2003. 

Program  Rebaseline.  As  a  result  of  schedule  delays,  the  Product  Manager  for 
SLAMRAAM  is  in  the  process  of  rebaselining  and  replanning  the  SLAMRAAM 
Program.  In  January  2006,  the  SLAMRAAM  Program  was  scheduled  for  an 
LRIP  decision  in  FY  2009,  but  the  LRIP  decision  has  now  slipped  to  FY  2010. 

The  SLAMRAAM  staff  stated  that  the  factors  causing  the  LRIP  decision  slip 
were  contractor  difficulties  in  following  established  engineering  processes, 
Congress  denying  requests  for  additional  funding  in  July  and  September  2006, 
and  funding  losses  resulting  from  the  U.S.  Marine  Corps  leaving  the 
SLAMRAAM  Program  in  September  2006. 

Funding  and  Contract  Data.  As  of  July  2007,  the  program’s  funding  to  develop 
and  procure  the  system  totaled  $622.5  million,  with  $208.3  million  in  research, 
development,  test,  and  evaluation  funds  and  $414.2  million  for  procurement  of 
hardware  (including  69  fire  units,  35  IFCSs,  55  AMRAAMs,  and  30  sensor  kits  to 
be  installed  on  the  Sentinel  Radar).  The  U.S.  Army  Aviation  and  Missile 
Command  awarded  a  contract  to  Raytheon  on  February  26,  2004,  for  $127  million 
to  develop  the  SLAMRAAM.  Through  November  2006,  the  Army  increased  the 
contract  value  to  $181.8  million  through  86  contract  modifications. 


Objectives 


The  audit  objective  was  to  evaluate  the  overall  management  of  the  Army 
SLAMRAAM  Program.  Because  the  program  is  in  the  system  development  and 
demonstration  phase,  we  determined  whether  management  was  cost  effectively 
preparing  the  program  for  the  LRIP  decision  program  review.  We  also  evaluated 
the  manager’s  internal  controls  as  they  related  to  the  audit  objectives.  See 
Appendix  A  for  a  discussion  of  the  audit  scope  and  methodology. 


Review  of  Internal  Controls 


We  detennined  that  material  internal  control  weaknesses  existed  in  the 
management  of  the  SLAMRAAM,  as  defined  by  DoD  Instruction  5010.40, 
“Managers’  Internal  Control  (MIC)  Program  Procedures,”  January  4,  2006.  The 
DoD  5000  series  of  guidance  requires  the  product  manager  to  exercise  discretion 


1  in  January  2005,  Program  Executive  Office  for  Air,  Space  and  Missile  Defense  merged  with  the  Program 
Executive  Office  for  Tactical  Missiles  to  become  the  Program  Executive  Office  for  Missiles  and  Space. 
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and  prudent  business  judgment  in  structuring  a  tailored,  responsive,  and 
innovative  program.  Planning  the  progression  of  SLAMRAAM  from 
development  into  LRIP  without  having  the  user  fully  defining  system  capability 
requirements,  completing  system  engineering  planning  requirements,  efficiently 
and  effectively  using  the  program  surveillance  resources  of  the  Defense  Contract 
Management  Agency  (DCMA),  and  planning  to  use,  as  directed,  unapproved 
information  assurance  guidance  comprise  less  than  prudent  business  practices. 
Implementing  our  recommendations  will  improve  internal  controls  by  ensuring 
that  the  Army  more  cost  effectively  and  efficiently  readies  the  SLAMRAAM 
program  for  LRIP.  We  will  provide  a  copy  of  this  report  to  the  senior  Army 
official  responsible  for  internal  controls  in  the  Department  of  the  Army. 
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A.  Defining  Capability  Requirements 


The  Director  of  Combat  Developments  (DCD),  U.S.  Army  Air  Defense 
Artillery  School  had  not  yet  sufficiently  defined  capability  requirements 
for  the  SLAMRAAM  in  the  draft  capability  production  document  (CPD) 
being  prepared  for  Army  staffing  to  support  the  LRIP  decision  program 
review  planned  for  FY  2010.  Specifically,  the  draft  CPD  did  not  identify 
system  effectiveness  as  a  key  performance  parameter,  and  duDioffid^ 
define  measurable  and  testable  canabilitv  requirements  for^^^^^^ 


le  snorttatts  in  denning  system  capabilities  in  tne  draft 
occurred  because  the  DCD  staff  did  not  adhere  to  established 
Chairman  of  the  Joint  Chiefs  of  Staff  (CJCS)  policy  for  defining  system 
capability  requirements.  Without  CPD  revision,  the  Army  will  be 
hindered  in  its  ability  to  ensure  that  the  SLAMRAAM  satisfies  essential 
warfighter  capability  requirements. 


Policies,  Procedures,  and  Guidance  for  Defining  Capability 
Requirements 


The  DoD  has  established  policies,  procedures,  and  guidance  for  defining 
capability  requirements  for  developing  weapon  systems. 

Policies  and  Procedures,  CJCS  Instruction  3 170. OIF,  “Joint  Capabilities 
Integration  and  Development  System,”  May  1,  2007,  and  CJCS  Instruction 
62 12.0  ID,  “Interoperability  and  Supportability  of  Information  Technology  and 
National  Security  Systems,”  March  8,  2006,  establish  DoD  policies  and 
procedures  for  defining  system  capability  requirements  through  the  Joint 
Capabilities  Integration  and  Development  System. 

Guidance.  CJCS  Manual  3 170.0 1C,  “Operation  of  the  Joint  Capabilities 
Integration  and  Development  System,”  May  1,  2007,  provides  guidance  and 
procedures  for  implementing  the  Joint  Capabilities  Integration  and  Development 
System. 


Defining  Requirements  in  the  Capability  Production 
Document 


As  of  July  2007,  DCD  was  developing  the  draft  “Capability  Production 
Document  for  SLARMRAAM  Increment  I,”  (undated)  for  Anny  staffing  to 
support  the  LRIP  decision  planned  for  FY  2010.  As  drafted,  the  CPD  did  not 
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identify  system  effectiveness  as  a  key  performance  parameter  nor  fully  define 
measurable  and  testable  capability  requirements. 

System  Effectiveness  as  a  Key  Performance  Parameter.  The  CJCS  Manual 
3 170.0 1C  states  that  key  performance  parameters  are  perfonnance  attributes  of  a 
system  considered  critical  or  essential  to  the  development  of  an  effective  military 
capability  and  make  a  significant  contribution  to  the  key  characteristics  of  the 
program.  The  manual  states  that  failure  to  meet  the  threshold  or  minimum 
performance  value  of  a  key  performance  parameter  may  result  in  management 
revaluating  the  program  or  modifying  production  increments. 


During  the  audit,  DCD  staff  stated  that  they  intended  for  system  effectiveness  to 
be  a  key  performance  parameter  but  that  the  SLAMRAAM  Product  Office 
questioned  this  intention.  DCD  staff  stated  that  the  SLAMRAAM  Product  Office 
questioned  establishing  system  effectiveness  as  a  key  perfonnance  parameter 
because  SLAMRAAM  system  effectiveness  was  directly  related  to  proven 


Because  system  effecti^nus^wn^xtremely  important  performance  attribute, 
combined  probability^ 

which  the  product  manager  can  mtiuence,  needs  to  Pe  mciudect  as  a  Key 
performance  parameter  in  the  CPD.  The  SLAMRAA^^oulcMull^neeUd^key 
performance  parameters  identified  in  the  draft  CPD. _ 

and  still 


cannot  meet 


tectiveness  requirements. 


Measurable  and  Testable  Capability  Requirements.  CJCS  Instruction 
3 170. OIF  requires  that  the  CPD  define  projected  capabilities  with  sufficient 
accuracy  to  begin  production.  Additionally,  the  CJCS  Manual  3 170.0 1C  states 
that  the  CPD  should  present  perfonnance  attributes  in  measurable  and  testable 
tenns.  However,  as  drafted,  the  CPD  did  not  contain  measurable  and  testable 
requirements  for  portions  of  two  key  perfonnance  parameters  and  six  other 
performance  attributes  that  are  identified  in  Appendix  B.  This  condition  occurred 
because  the  DCD  staff  did  not  adhere  to  guidance  in  the  CJCS  Manual  3 170.0 1C 
for  defining  key  performance  parameters  and  perfonnance  attributes.  DCD  still 
has  time  to  better  define  these  capability  requirements  because  it  still  has  not 
forwarded  the  draft  CPD  to  the  U.S.  Anny  Training  and  Doctrine  Command  for 
review.  Appendix  B  provides  descriptions  of  the  key  performance  parameters 
and  performance  attributes  that  DCD  needs  to  define  in  measurable  and  testable 
tenns  in  the  draft  CPD. 


Conclusion 


Imprecisely  defined  capability  requirements  in  the  draft  CPD  would  hinder  the 
Anny  in  its  ability  to  ensure  that  the  SLAMRAAM  satisfies  essential  warfighter 
caoabilitvrcauircmcntsbcIbrcthcLRIPdccisioirB^dcnti  lying  the  probability 

/  Z  I  ... 

parameter,  rather  than  a  perlormance  attribute,  Army  acquisition  managers  can 
increase  the  value  given  to  system  performance  when  making  the  LRIP  program 
decision.  Further,  defining  capability  requirements  in  the  CPD  in  measurable  and 
testable  terms  will  enable  the  Army  to  fully  verily  that  the  SLAMRAAM  will 
provide  the  capabilities  the  warfighter  needs  before  the  LRIP  program  decision. 


Management  Comments  on  the  Finding  and  Audit  Response 

Summaries  of  management  comments  on  the  finding  and  our  response  are  in 
Appendix  C. 


6 

FOR  OFFICIAL  USE  ONLY 


b(3) 


Recommendation,  Management  Comments,  and  Audit 
Response 


A.  We  recommend  that  the  Director  of  Combat  Developments,  U.S.  Army 
Air  Defense  Artillery  School  revise  the  draft  capability  production  document 
for  the  Surface-Launched  Advanced  Medium-Range  Air-to-Air  Missile  in 
accordance  with  policy  in  Chairman  Joint  Chiefs  of  Staff  Instruction 
3170.01F,  “Joint  Capabilities  IntegrcitioiymcHIevelogmeirtS^stemj^^la^l, 

Hefine  the  probability 

as  a  key  performance  parameterj_anfl  to  estat 
_ anabilitv  requirements  for^ 


measural 


and 


Director  of  Combat  Developments  Comments.  The  Director  of  Combat 
Developments,  U.S.  Army  Air  Defense  Artillery  School  partially  concurred, 
stating  that  implementing  the  recommendations  would  address  the  reported 
weaknesses  in  capability  development.  SpecificallyAh^Iirectoi^oncungcUUth 

revising  the  draft  CPD  to  define  the  probabilities  off” . 

as  a  key  perfonnance  parameter  ana  to  establish 
measurable  ana  testable  capability  requirements  fori 

concurred  with 


Audit  Response.  The  director’s  planned  revisions  to  the  draft  CPD  are 
responsive  to  the  recommendation. 


Project  Manager,  Cruise  Missile  Defense  System  Comments.  Although  not 
recjuirecUc^ommun^h^rojec^Manager,  CMDS  stated  that,  while  BBT 

do  contribute  to  system  effectiveness,  they 
lo  not  nee^^^^e^enonnanc^aramete^The  project  manager  stated  that 

making  a 

would  b^tmna^maRalu^mc^n^Ann^iaHiot  demonstrated  the  performance 
of  these  elements  within  a  SLAMRAAM  system.  Thus,  the  project  manager 
stated  that  system  effectiveness  should  remain  a  perfonnance  attribute  that  the 
Anny  will  quantify  at  the  LRIP  decision  as  end-to-end  performance,  including  the 
sensor  and  missile  perfonnance.  The  project  manager  further  stated  that  he 
agreed  that  the  CPD  should  state  all  capability  requirements  in  measurable, 
testable  terms. 


Audit  Response.  As  we  discuss  in  the  finding,  CJCS  Manual  3170.01C  states 
that  key  performance  parameters  are  critical  or  essential  performance  attributes  in 
the  development  of  an  effective  military  capability  and  significantly  contribute  to 
the  key  characteristics  of  a  program.  Because  system  effectiveness  is  essential  to 
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developing  a  military  capability  for  th^S^AMi^A^^hyaroiecUnanagek^^ 

statement  that  the  performance  of  the 

has  not  been  demonstrated  within  a  S^ffl^I^Hj^systenHioe^ioRuppo^^^ 
keeping  systems  effectiveness  as  a  performance  attribute.  Instead,  the  project 
manager’s  comments  could  support  an  argument  to  make  overall  system 
effectiveness  a  key  perfonnance  parameter,  since  the  Army  has  not  demonstrated 
the  perfonnance  of  the  SLAMRAAM  system. 
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B.  Planning  for  Systems  Engineering 

The  Product  Manager  for  SLAMRAAM  was  in  the  process  of  developing 
a  systems  engineering  plan  (SEP)  to  support  its  systems  engineering 
management  approach  for  SLAMRAAM.  In  the  meantime,  the  product 
manager  was  using  the  contractor’s  systems  engineering  management  plan 
(SEMP)  that  did  not  include  key  information  needed  to  effectively  manage 
the  contractor’s  systems  engineering  efforts.  Specifically,  the  SEMP  did 
not  define  entrance  and  success  criteria  for  planned  technical  reviews. 
Additionally,  the  SEMP  did  not  fully  define  the  systems  engineering 
processes  the  product  manager  was  applying.  These  conditions  occurred 
because  the  product  manager  did  not  timely  implement  DoD  guidance  for 
developing  a  SEP  to  manage  its  systems  engineering.  As  a  result,  the 
product  manager  did  not  avail  the  program  office  with  a  systems 
engineering  management  approach  needed  to  cost  effectively  manage, 
develop,  and  integrate  the  system  to  fully  meet  program  requirements. 


Systems  Engineering  Policy  and  Guidance 


Policy.  DoD  acquisition  systems  engineering  policy  is  provided  in  DoD 
Directive  5000.1,  “The  Defense  Acquisition  System,”  May  12,  2003;  DoD 
Instruction  5000.2,  “Operation  of  the  Defense  Acquisition  System,”  May  12, 

2003;  and  in  two  memorandums  that  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  [USD(AT&L)]  issued  in  2004  to  clarify 
systems  engineering  policy.  The  two  policy  memorandums  are  planned  for 
inclusion  in  the  next  update  of  DoD  Instruction  5000.2. 

DoD  Directive  5000.1.  The  directive  requires  that  program  managers 
apply  a  systems  engineering  approach  that  optimizes  system  performance  and 
minimizes  ownership  costs  to  manage  acquisition  programs. 

DoD  Instruction  5000.2.  The  instruction  emphasizes  the  requirement  for 
using  systems  engineering  to  operate  and  support  the  system  in  the  most  cost- 
effective  manner  over  its  total  life  cycle.  It  states  that  the  effective  sustainment  of 
weapon  systems  begins  with  the  design  and  development  of  reliable  and 
maintainable  systems  through  the  continuous  application  of  a  robust  systems 
engineering  methodology. 

USD(ATL)  Memorandum,  “Policy  for  Systems  Engineering  in  DoD,” 
February  20,  2004.  The  memorandum  requires  that  program  managers  for  all 
acquisition  programs  develop  a  SEP  integrated  with  the  acquisition  strategy  for 
all  milestone  decisions.  Specifically,  the  memorandum  states  that  all  programs, 
regardless  of  acquisition  category,  shall  apply  a  robust  systems  engineering 
approach  that  balances  total  system  perfonnance  and  total  ownership  costs  within 
the  family-of-systems  context.  The  SEP  should  describe  the  program’s  overall 
technical  approach,  applicable  perfonnance  incentives,  and  success  criteria  for 
technical  reviews. 
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USD(ATL)  Memorandum,  “Policy  Addendum  for  Systems 
Engineering,”  October  22,  2004.  The  memorandum  requires  that  the  SEP 
contain  event-driven,  rather  than  schedule-driven,  criteria  for  holding  technical 
reviews  that  evaluate  program  progress.  Specifically,  the  memorandum  requires 
that  program  managers  conduct  technical  reviews  only  when  the  system  under 
development  meets  the  review  entrance  criteria  documented  in  the  SEP. 

Guidance.  In  addition  to  the  above  policy,  the  USD(AT&L)  issued  the  guidance 
memorandum,  “Implementing  Systems  Engineering  Plans  in  DoD  -  Interim 
Guidance,”  March  30,  2004,  to  help  program  managers  in  preparing  SEPs.  The 
memorandum  recommends  that  program  managers  establish  the  SEP  early  in  the 
program’s  life  cycle  to  guide  all  technical  aspects  of  an  acquisition  program  and 
then  update  the  plan  as  the  program  progresses  through  the  acquisition 
milestones.  In  addition,  the  memorandum  states  that  the  SEP  supports  program 
management  by  defining  comprehensive  Government  and  contractor  systems 
engineering  activities  and  responsibilities.  The  memorandum  further  states  that 
the  SEP  describes  the  program’s  overall  technical  approach  by  defining  the 
systems  engineering  processes,  metrics,  use  of  technical  reviews,  and  criteria  for 
successful  technical  reviews. 


Developing  the  Systems  Engineering  Plan 


Early  during  the  audit,  in  November  2006,  SLAMRAAM  Product  Office 
personnel  stated  they  did  not  have  a  SEP  to  support  the  systems  engineering 
process  for  the  SLAMRAAM  in  accordance  with  USD(AT&L)  guidance.  The 
SLAMRAAM  Product  Office  staff  stated  that  while  they  had  started  working 
towards  a  SEP,  they  did  not  have  a  draft  version  of  the  SEP  for  us  to  review.  In 
the  meantime,  the  SLAMRAAM  Product  Office  staff  was  using  the  contractor- 
developed  “Surfaced  Launched  AMRAAM  (SLAMRAAM)  Systems  Engineering 
Management  Plan  (SEMP),”  October  10,  2005,  to  support  the  systems 
engineering  process  for  the  program.  This  SEMP  did  not  fully  meet  the 
requirements  of  the  SEP  because  it  did  not  include  key  information  that  DoD 
systems  engineering  policy  and  guidance  requires  or  encourages  project  managers 
to  define.  Specifically,  the  SEMP  did  not  include  event-driven  entrance  and 
success  criteria  for  planned  technical  reviews,  as  required  respectively  by  the 
October  22,  2004,  and  the  February  20,  2004,  USD(AT&L)  policy 
memorandums.  Additionally,  the  SEMP  did  not  fully  define  systems  engineering 
processes  that  the  product  manager  was  applying  as  specified  in  the  March  2004 
USD(AT&L)  memorandum  regarding  the  contents  of  the  SEP.  The  limitations  of 
the  contractor-developed  SEMP  as  compared  to  the  requirements  in  the  DoD 
policy  and  guidance  are  discussed  below. 

Technical  Reviews.  The  SEMP  did  not  define  event-driven  entrance  and  success 
criteria  for  planned  technical  reviews,  including  a  system  requirements  review,  a 
system  design  review,  a  critical  design  review,  and  a  test  readiness  review. 
Technical  reviews  measure  contractor  progress  toward  attaining  well-defined 
technical,  cost,  and  schedule  goals;  assess  system  technical  maturity  and  risk;  and 
provide  infonnation  to  support  program  decisions.  Including  entrance  and 
success  criteria  for  the  technical  reviews  specified  in  the  SEMP  would  help  the 
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product  manager  to  better  ensure  that  technical  reviews  are  held  only  when 
program  accomplishments  make  them  warranted  and  meaningful  in  measuring 
program  results. 

Systems  Engineering  Process.  The  contractor-developed  SEMP  did  not  fully 
define  all  systems  engineering  processes  applied  to  the  SLAMRAAM. 
Specifically,  the  SEMP  only  identified  the  processes  that  the  contractor  initially 
planned  to  complete  during  the  system  development  and  demonstration  phase  of 
the  acquisition  process  and  the  contractor  had  not  updated  it  to  implement  the 
results  of  the  completed  engineering  processes,  such  as  the  system  requirements 
review.  Prepared  and  validated  as  required,  the  product  office  SEP  would  have 
identified  key  systems  engineering  processes  during  the  system  development  and 
demonstration  phase,  included  updates  based  on  completed  technical  reviews,  and 
described  how  the  systems  engineering  process  would  support  the  technical 
products  of  each  acquisition  phase. 


Product  Manager  Actions  Since  November  2006 

The  SLAMRAAM  Product  Manager  prepared  a  draft,  “Surface  Launched 
Advanced  Medium  Air-to-Air  Missile  System  (SLAMRAAM)  Systems 
Engineering  Plan  (SEP)  Revision  0  Supporting  Milestone  C,”  that  was  dated 
June  18,  2007.  As  written,  however,  the  draft  SEP  did  not  comply  with 
USD(ATL)  policy  and  guidelines  because  it  did  not  include  event-driven 
technical  reviews,  with  entrance  and  success  criteria  and  did  not  fully  describe  the 
systems  engineering  processes  applied  to  the  SLAMRAAM. 


Factors  Affecting  Development  of  the  SEP 


Because  the  system  development  and  demonstration  decision  for  SLAMRAAM 
was  held  in  September  2003  and  pre-dated  the  February  2004  requirements  to 
develop  a  SEP,  the  Product  Manager,  SLAMRAAM  chose  to  rely  on  the 
contractor’s  SEMP  to  manage  the  systems  engineering  process.  Since 
February  2004,  the  product  manager  did  not  respond  to  USD(AT&L)  guidance 
issued  in  2004  by  timely  and  proactively  establishing  a  SEP  early  in  the 
program’s  life  cycle  to  guide  all  technical  aspects  of  the  program. 


Conclusion 


As  a  result  of  not  developing  a  SEP  to  manage  systems  engineering,  the  product 
manager  did  not  avail  the  program  office  with  a  systems  engineering  management 
approach  needed  to  cost  effectively  manage,  develop,  and  integrate  the  system  to 
fully  meet  program  requirements.  While  the  February  20,  2004,  USD(AT&L) 
policy  memorandum  did  not  require  the  product  manager  to  complete  a  SEP  until 
the  next  acquisition  milestone  review  (LRIP  is  planned  in  FY  2010  for 
SLAMRAAM),  the  March  30,  2004,  guidance  memorandum  clearly  explained  the 
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benefits  of  preparing  a  SEP  early  in  the  program’s  life  cycle.  Specifically,  a  SEP 
would  provide  the  product  manager  with  a  well-defined  systems  engineering 
process  for  the  SLAMRAAM.  In  addition,  a  SEP  would  include  event-driven 
technical  reviews  with  defined  entrance  and  success  criteria,  thus  helping  the 
product  manager  to  more  effectively  manage  the  systems  engineering  process. 


Management  Comments  on  the  Finding  and  Audit  Response 


Summaries  of  management  comments  on  the  finding  and  our  response  are  in 
Appendix  C. 


Recommendation,  Management  Comments,  and  Audit 
Response 


B.  We  recommend  that  the  Product  Manager,  Surface-Launched  Advanced 
Medium-Range  Air-to-Air  Missile  revise  the  draft  systems  engineering  plan 
to  establish  a  comprehensive  systems  engineering  plan  (in  accordance  with 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
memorandums,  “Policy  for  Systems  Engineering  in  DoD,”  February  20, 

2004;  “Policy  Addendum  for  Systems  Engineering,”  October  22,  2004;  and 
“Implementing  Systems  Engineering  Plans  in  DoD  -  Interim  Guidance,” 
March  30,  2004)  that  includes  planning  for  technical  reviews  that  are  event- 
driven,  with  entrance  and  success  criteria,  and  fully  describes  the  systems 
engineering  processes  applied  to  the  Surface-Launched  Advanced  Medium- 
Range  Air-to-Air  Missile. 

Management  Comments.  The  Project  Manager,  Cruise  Missile  Defense  System, 
responding  for  the  Product  Manager,  Surface-Launched  Advanced  Medium- 
Range  Air-to-Air  Missile,  concurred.  He  stated  that  the  SEP  has  been  and  will 
continue  to  be  prepared  in  accordance  with  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  memorandums:  “Policy  for  Systems 
Engineering  in  DoD,”  February  20,  2004;  “Policy  Addendum  for  Systems 
Engineering,”  October  22,  2004;  and  “Implementing  Systems  Engineering  Plans 
in  DoD  -  Interim  Guidance,”  March  30,  2004. 

Audit  Response.  The  project  manager’s  commitment  to  preparing  the  SEP  in 
accordance  with  applicable  policy  and  guidance  is  responsive  to  the 
recommendation. 
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C.  Establishing  Defense  Contract 
Management  Agency  Support  Responsibilities 

The  Commander,  Defense  Contract  Management  Agency  (DCMA),  Space 
Sensors  and  Communications  Operations,  Raytheon  Integrated  Defense 
Systems  and  the  Project  Manager,  Cruise  Missile  Defense  System 
(CMDS)  did  not  adequately  establish  and  document  needed  DCMA 
support  in  the  memorandum  of  agreement  (MO A)  for  the  SLAMRAAM 
Program.  Specifically,  the  MOA  did  not  reference  current  DCMA  policy 
and  fully  establish  links  between  the  MOA  performance  metrics  and  the 
project  manager’s  desired  outcomes  for  the  SLAMRAAM  Program. 
Additionally,  the  commander  did  not  fully  describe  DCMA  activities  to 
support  the  SLAMRAAM  Program  in  a  surveillance  plan  and  establish 
letters  of  delegation  (LOD)  with  other  DCMA  contract  management 
offices  for  surveillance  at  three  of  four  SLAMRAAM  subcontractors. 
These  conditions  occurred  because  the  commander  did  not  completely 
adhere  to  provisions  in  the  Federal  Acquisition  Regulation,  the  DCMA 
Instruction,  and  the  DCMA  Guidebook  for  preparing  an  MOA, 
formulating  a  surveillance  plan,  and  establishing  LODs.  As  a  result, 
DCMA  did  not  have  infonnation  needed  to  provide  the  project  manager 
with  the  most  informed  recommendations  regarding  contractor  progress 
toward  attaining  contract  cost,  schedule,  and  performance  requirements. 


Regulations  and  Guidance  for  Defense  Contract  Management 
Agency  Support 


Federal  and  DCMA  regulations  and  guidance  define  the  DCMA  role  in 
supporting  the  program  manager’s  development  of  weapon  systems. 

Federal  Acquisition  Regulation,  Federal  Acquisition  Regulation  42.302, 
“Contract  Administration  Functions,”  specifies  the  contract  administration 
functions  that  Federal  organizations  normally  delegate  to  contract  administration 
offices.  Those  contract  administration  functions  include  program  status  reporting; 
assessing  contractor  compliance  with  contract  terms;  surveilling  contractor 
engineering  efforts  and  management  systems;  and  reviewing  and  evaluating  the 
contractor’s  logistic  support,  maintenance,  and  modification  programs. 

DCMA  Policy  and  Guidance.  The  DCMA  Instruction  and  Guidebook  provide 
mandatory  policy  and  guidance  for  performing  the  contract  management 
functions  listed  in  the  Federal  Acquisition  Regulation.  Specifically,  they  provide 
the  DCMA  staff  with  direction  for  perfonning  outcome-based  program 
management  support  for  DoD  acquisition  programs,  including  direction  for: 

•  establishing  MOAs  with  program  managers  that  focus  on  desired 
program  outcomes, 
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•  establishing  surveillance  plans  detailing  the  tasks  necessary  to  meet 
the  provisions  of  the  MOA,  and 

•  establishing  and  managing  program  support  teams  led  by  program 
integrators  to  carry  out  the  tasks  documented  in  the  surveillance  plan. 


Defense  Contract  Management  Agency  Support  for 
SLAMRAAM 


The  DCMA  provided  the  Project  Manager,  CMDS  with  program  status  reports  in 
accordance  with  the  May  2006  MOA.  However,  the  commander  and  the  project 
manager  did  not  adequately  establish  and  document  needed  DCMA  support  in  the 
May  2006  MOA  for  the  SLAMRAAM  Program.  Specifically,  the  MOA  did  not 
reference  the  current  DCMA  regulation  and  guidance  or  include  required  links 
between  the  MOA  performance  metrics  and  the  project  manager’s  desired 
program  outcomes.  Additionally,  the  commander  did  not  fully  describe  DCMA 
activities  to  support  the  program  in  a  surveillance  plan  and  establish  LODs  with 
other  contract  management  offices  for  surveillance  at  three  subcontractors. 

Preparing  the  Memorandum  of  Agreement,  The  approved  “Memorandum  of 
Agreement  Between  the  Cruise  Missile  Defense  Systems  (CMDS)  Project  Office, 
Redstone  Arsenal,  AL  and  the  DCMA  Raytheon  Integrated  Defense  Systems  - 
Tewksbury  and  Andover,  MA,”  May  1,  2006,  did  not  confonn  to  the  DCMA 
Instruction.  Specifically,  the  MOA  referenced  the  expired  One  Book2  rather  than 
the  current  DCMA  Instruction  and  DCMA  Guidebook,  and  did  not  include  an 
annex  showing  the  cause-and-effect  analysis  that  links  each  MOA  performance 
metric  and  standard  to  the  customer  desired  outcomes. 

Applicable  Policy  and  Guidance.  On  May  24,  2004,  DCMA  transitioned 
from  the  DCMA  One  Book  to  the  DCMA  Instruction  and  DCMA  Guidebook  for 
performing  contract  management  functions.  DCMA  officials  stated  that  the  new 
DCMA  program  guidance  allowed  DCMA  to  implement  performance-based 
management  in  accomplishing  its  program  support  function.  The  DCMA  officials 
stated  that  performance-based  management  enables  DCMA  to  more  efficiently 
use  limited  resources  to  support  program  managers  by  establishing  performance 
metrics  and  standards  in  the  MOA  to  focus  and  measure  DCMA’s  efforts  in 
helping  customers  meet  their  desired  outcomes.  Accordingly,  the  commander  and 
the  project  manager  need  to  update  the  MOA  to  reference  and  apply  the 
appropriate  DCMA  policy  and  guidance  to  more  efficiently  use  resources  in 
support  of  the  SLAMRAAM  Program. 

Cause-and-Effect  Analysis  Annex.  The  DCMA  Instruction  requires 
MO  As  to  include  an  annex  that  documents  a  cause-and-effect  analysis  to  link 
MOA  performance  metrics  and  standards  to  customer-desired  outcomes.  The 
MOA  did  not  include  a  cause-and-effect  analysis  that  linked  performance  metrics 
to  customer-desired  outcomes  for  the  SLAMRAAM  Program.  As  a  result, 


2  The  DCMA  One  Book  that  contained  DCMA  policy  was  cancelled  in  May  2004. 
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DCMA  could  better  support  the  program  manager  by  focusing  its  limited 
resources  on  performance  metrics  and  standards  to  help  the  customer  achieve  its 
desired  outcomes  and  enable  the  project  manager  to  provide  informed 
recommendations  on  contractor  performance. 

Describing  Activities  to  Support  Program.  The  DCMA  Guidebook 
recommends  that  surveillance  plans,  as  part  of  the  MOA,  prioritize  DCMA 
support  of  an  acquisition  program,  and  identify  what,  when,  where,  and  how  the 
DCMA  staff  will  support  the  program  manager.  The  DCMA  Instruction  also 
recommends  that  the  MOA  include  performance  metrics,  performance  standards, 
and  activities  to  support  surveillance  in  activity  annexes  in  the  MOA.  While  the 
MOA  contained  perfonnance  metrics  and  activities  needed  to  support  DCMA 
surveillance,  it  did  not  prioritize  DCMA  efforts  in  support  of  the  SLAMRAAM 
Program,  include  performance  standards  to  implement  the  performance  metrics, 
nor  specify  who  was  responsible  for  specific  activities. 

Actions  to  Improve  the  Quality  of  Surveillance  Plans.  To  help  DCMA 
contract  management  offices  plan  and  track  surveillance  support  activities, 
DCMA  Headquarters  was  developing  the  Electronic  Data  Management  System,  a 
Web-based  management  tool  that  will  link  surveillance  plan  policies,  processes, 
methods,  standards,  and  controls  to  assure  accountability,  integration,  and 
configuration  control.  DCMA  staff  stated  that  contract  management  offices  will 
be  required  to  use  the  Electronic  Data  Management  System,  but  will  be  able  to 
tailor  the  surveillance  plan  for  their  acquisition  programs. 

DCMA  was  developing  and  integrating  the  Electronic  Data  Management  System 
in  three  phases.  In  the  first  phase,  DCMA  will  store  commitments,  resource 
requirements,  and  program  objective  memorandum  data.  In  the  second  phase, 
DCMA  will  distribute  the  application  to  contract  management  offices  and  refine 
deficiencies.  In  the  third  phase,  DCMA  headquarters  will  set  policies, 
procedures,  and  controls  for  the  integration  of  all  management  systems  and 
databases.  Further,  DCMA  will  integrate  the  systems  and  databases  to  support 
contract  management  office  surveillance  of  acquisition  programs  by  including 
applications  that  record  performance  metrics,  performance  commitments,  and 
activities.  DCMA  staff  stated  that  implementation  of  the  three  phases  will  take  at 
least  5  years. 

Because  full  implementation  of  the  Electronic  Data  Management  System  is  at 
least  5  years  away,  DCMA  plans  to  continue  using  surveillance  plans  or  activity 
annexes  as  part  of  MOAs  to  describe  surveillance  activities  to  support  acquisition 
programs.  Although  DCMA  plans  to  require  contract  management  offices  to  use 
the  Electronic  Data  Management  System  to  link  surveillance  activities  for  all 
acquisition  programs  supported  under  MOAs,  DCMA  did  not  establish  a 
mandatory  requirement  for  contract  management  offices  to  use  surveillance  plans 
or  activity  annexes  in  the  interim.  To  ensure  that  program  offices  obtain  needed 
contractor  administration  surveillance  support,  DCMA  needs  to  require  contract 
management  offices  to  include  a  surveillance  plan  or  activity  annexes,  tailored  to 
the  acquisition  program  supported,  as  part  of  all  MOAs  until  the  Electronic  Data 
Management  System  is  implemented.  Without  a  tailored  surveillance  plan  or 
activity  annexes  in  the  MOA,  contract  management  offices  cannot  ensure  that 
DCMA  resources  are  used  effectively  to  achieve  customer-desired  outcomes. 
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Establishing  Letters  of  Delegation.  The  DCMA  Instruction  states  that  LODs 
will  specify  the  administrative  requirements,  perfonnance  commitments,  and 
support  between  DCMA  contract  management  offices.  The  DCMA  staff  at  Space 
Sensors  and  Communications  Operations  did  not  establish  LODs  to  support  the 
SLAMRAAM  Product  Manager  with  DCMA  contract  management  offices 
located  at  or  near  three  of  the  four  SLAMRAAM  subcontractor  facilities. 

•  Boeing  Huntsville.  Boeing  Huntsville  developed  the  prototype  IFCS. 
DCMA  staff  stated  that  the  commander  did  not  approve  a  draft  LOD 
with  DCMA,  Boeing  Huntsville  because  it  was  not  written  in 
customer-desired  outcome  terms.  In  February  2007,  the  DCMA 
Program  Integrator  for  SLAMRAAM  stated  that  Boeing  had 
completed  the  current  contract,  as  Boeing  had  delivered  the  five  IFCS 
prototypes  under  contract.  The  original  contract  cost  for  developing 
the  IFCS  prototypes  was  $18.9  million  and  the  actual  cost  was 

$31.5  million. 

•  Raytheon  Network  Centric  Systems.  Raytheon  Network  Centric 
Systems  was  developing  the  software  for  integration  of  the  enhanced 
position  location  reporting  system  radio.  DCMA  staff  stated  that  an 
LOD  was  not  established  with  Raytheon  Network  Centric  because  the 
subcontractor  was  developing  software  for  use  on  Government- 
furnished  equipment.  DCMA  staff  stated  that  Raytheon  Network 
Centric  Systems  cost  at  completion  was  $4.2  million,  but  were  unable 
to  provide  documentation  to  support  this  statement. 

•  Thales  Raytheon  Systems.  Thales  Raytheon  Systems  was  developing 
software  for  integration  of  the  Sentinel  Radar  as  part  of  the 
SLAMRAAM.  DCMA  staff  stated  that  an  LOD  was  not  established 
with  Thales  Raytheon  Systems  because  the  subcontractor  was  also 
developing  software  for  use  on  Government-furnished  equipment. 
DCMA  stated  that  the  cost  at  completion  was  $4.0  million,  but  were 
unable  to  provide  documentation  to  support  this  statement. 

With  regard  to  the  Boeing  subcontract,  the  DCMA  Program  Integrator  for 
SLAMRAAM  stated  that  through  an  LOD  she  could  have  provided  more 
complete  reporting  and  analysis  on  the  IFCS  to  the  Project  Manager,  CMDS. 
Specifically,  while  a  DCMA  support  program  integrator  at  Boeing  Huntsville 
stated  that  he  provided  the  project  manager  with  regular  reports  and  analysis  of 
Boeing’s  progress  on  the  IFCS,  he  stated  that,  without  an  approved  LOD,  his 
reporting  was  informal  and  did  not  include  earned  value  management  cost  and 
schedule  analysis.  Although  the  Raytheon  subcontractors  were  working  on 
Government-furnished  equipment,  they  were  developing  the  software  for 
integration  of  the  equipment  in  the  SLAMRAAM  Program.  Therefore,  we 
believe  that  the  formalized  reporting,  to  include  earned  value  management,  that 
the  DCMA  support  program  integrator  at  Boeing  Huntsville  could  have  provided 
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under  an  LOD  would  have  given  the  project  manager  more  meaningful 
information  on  the  subcontractors’  progress  towards  satisfying  SLAMRAAM 
cost,  schedule,  and  performance  requirements. 

The  staff  at  DCMA  Space  Sensors  and  Communications  Operations  did  establish 
an  LOD  with  DCMA  International  Northern  Europe  to  oversee  the  development 
of  software  by  Kongsberg  Defence  and  Aerospace.  As  a  result,  the  DCMA 
Program  Integrator  for  SLAMRAAM  was  kept  up-to-date  on  Kongsberg  Defence 
and  Aerospace’s  cost,  schedule,  and  performance  against  contract  requirements. 


Conclusion 


Without  a  focused  and  comprehensive  MO  A,  supported  by  a  surveillance  plan 
and  LODs,  DCMA  staff  were  not  in  a  position  to  provide  the  Project  Manager, 
CMDS  with  the  most  timely  and  meaningful  insights  and  recommendations 
regarding  the  contractor  progress  toward  attaining  cost,  schedule,  and 
performance  contract  requirements  for  SLAMRAAM.  The  project  manager  needs 
this  information  to  make  informed  decisions  concerning  the  SLAMRAAM 
Program. 


Management  Comments  on  the  Finding  and  Audit  Response 

Summaries  of  management  comments  on  the  finding  and  our  response  are  in 
Appendix  C. 


Recommendations,  Management  Comments,  and  Audit 
Response 


C.l.  We  recommend  that  the  Commander,  Defense  Contract  Management 
Agency  and  the  Project  Manager,  Cruise  Missile  Defense  Systems  revise  the 
memorandum  of  agreement  for  the  Surface-Launched  Advanced  Medium- 
Range  Air-to-Air  Missile  to: 

a.  Reference  the  current  Defense  Contract  Management  Agency 
Instruction  and  Defense  Contract  Management  Agency  Guidebook;  and 

b.  Include  an  annex  that  documents  a  cause-and-effect  analysis 
linking  Defense  Contract  Management  Agency  performance  metrics  and 
standards  to  the  customer-desired  outcome,  in  accordance  with  the  Defense 
Contract  Management  Agency  Instruction. 

Defense  Contract  Management  Agency  Comments.  The  Acting  Director, 
Defense  Contract  Management  Agency  responding  for  the  Commander, 
concurred.  He  stated  that  DCMA,  Space  Sensors  and  Communications 
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Operations,  Raytheon  Integrated  Defense  Systems  was  updating  the  MOA  in 
accordance  with  the  current  agency  guidelines  and  instructions.  The  acting 
director  estimated  that  the  MOA  would  be  updated  and  completed  by  the  first 
quarter  of  FY  2008. 

Army  Comments.  The  Project  Manager,  Cruise  Missile  Defense  System, 
responding  for  the  Product  Manager,  Surface-Launched  Advanced  Medium- 
Range  Air-to-Air  Missile,  concurred,  stating  that  CMDS  and  DCMA  were 
establishing  a  new  MOA,  based  on  2007  DCMA  guidance,  which  will  reflect  the 
more  recent  changes  DCMA  has  made  to  the  performance-based  management 
process  since  the  MOA  was  issued  in  May  2006. 

Audit  Response.  The  acting  director’s  and  project  manager’s  plans  to  establish 
an  updated  MOA,  based  on  current  DCMA  guidelines  and  instructions,  are 
responsive  to  the  recommendation. 

C.2.  We  recommend  that  the  Acting  Director,  Defense  Contract 
Management  Agency  update  the  Defense  Contract  Management  Agency 
Instruction  to  require  that  memorandums  of  agreement  with  acquisition 
program  managers  to  include  surveillance  plans  or  activity  annexes  that 
prioritize  Defense  Contract  Management  Agency  support;  performance 
metrics  and  standards;  and  identify  what,  when,  where,  and  how  the  Defense 
Contract  Management  Agency  staff  provides  surveillance  until  the 
Electronic  Data  Management  System  is  completed. 

Management  Comments.  The  Acting  Director,  Defense  Contract  Management 
Agency  nonconcurred.  He  stated  that  DCMA  policy  does  not  require  the 
inclusion  of  the  performance  commitment  strategy  (surveillance  plan)  in  the 
MO  As  they  execute  with  their  customers,  because  their  commitment  is  not  to  the 
activities  that  make  up  the  strategy,  but  to  the  results  noted  in  the  body  of  the 
MOA.  He  stated  that,  if  DCMA  made  the  inclusion  of  Annex  E  mandatory,  then 
DCMA  would  be  committed  not  only  to  the  desired  results,  but  to  a  strategy  that 
may  need  to  change,  if  analysis  shows  that  progress  toward  the  desired  results 
was  not  satisfactory.  He  stated  that,  while  the  inclusion  of  Annex  E  is  not 
mandatory  in  the  MOA,  DCMA  does  require  that  contract  management  offices 
execute  strategies  to  achieve  their  performance  commitments.  Therefore,  he 
stated  that  DCMA  chose  to  use  strategies  contained  in  the  individual  performance 
plans  of  its  functional  specialists  to  ensure  execution  of  the  activities  needed  to 
achieve  perfonnance  commitments.  By  documenting  the  strategies  in  this 
fashion,  the  acting  director  stated  that  DCMA  makes  execution  a  direct  linkage 
between  the  organizational  perfonnance  commitments  and  the  factors  that 
determine  performance  of  the  DCMA  workforce.  He  stated  that  this  linkage  of 
organizational  and  individual  perfonnance  makes  the  DCMA’s  performance- 
based  management  approach  focused  on  appropriate  priorities  and  “self- 
governing.” 

Audit  Response.  The  acting  director’s  comments  did  not  adequately  address  the 
recommendation.  The  use  of  surveillance  plans  or  activity  annexes  offers 
significant  advantages  over  using  the  strategies  contained  in  the  individual 
performance  plans  of  the  functional  specialists  to  ensure  that  the  specialists 
execute  the  activities  needed  to  achieve  perfonnance  commitments.  The 
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surveillance  plan,  or  activities  annex,  enhances  the  MOA  by  providing  an  overall 
consolidated  strategy  for  the  DCMA  program  integration  staff,  including 
functional  specialists,  to  follow  in  perfonning  required  support  functions. 
Conversely,  using  the  individual  performance  plans  of  functional  specialists 
fragments  strategies  into  the  contributions  of  individual  functional  specialists.  As 
a  further  complication,  individual  functional  specialists  often  support  multiple 
acquisition  programs,  each  with  a  different  MOA  containing  tailored  performance 
commitments.  Additionally,  if  analysis  shows  that  program  progress  toward  the 
desired  results  is  not  satisfactory  and  strategy  changes  are  needed,  the  DCMA 
would  have  to  change  multiple  individual  performance  plans,  versus  one 
surveillance  plan  or  activity  annex.  In  summary,  using  the  surveillance  plan  or 
activity  annex  as  part  of  the  MOA  provides  the  DCMA  Program  Integrator  with  a 
more  efficient  means  for  ensuring  that  functional  specialists  perform  necessary 
activities  to  meet  commitments  to  acquisition  program  managers.  As  an 
additional  consideration,  the  acting  director’s  comments  to  the  above 
recommendation  are  not  consistent  with  his  comments  on  a  recommendation  in 
another  recent  DoD  Inspector  General  Report.  Specifically,  in  DoD  Inspector 
General  Report  No.  D-2007-084,  “Acquisition  of  the  Navy  Rapid  Airborne  Mine 
Clearance  System,”  April  11,  2007,  the  acting  director  concurred  with  a 
recommendation  to  establish  a  surveillance  plan  for  the  Rapid  Airborne  Mine 
Clearance  System  Program.  Accordingly,  we  request  that  the  acting  director 
reconsider  his  position  and  provide  comments  in  response  to  the  final  report. 

C.3.  We  recommend  that  the  Commander,  Defense  Contract  Management 
Agency,  Space  Sensors  and  Communications  Operations,  Raytheon 
Integrated  Defense  Systems  establish  letters  of  delegation  with  supporting 
Defense  Contract  Management  Agency  contract  management  offices  for 
subcontractor  surveillance  at  Raytheon  Network  Centric  Systems  and  Thales 
Raytheon  Systems  subcontractors,  in  accordance  with  the  Defense  Contract 
Management  Agency  Instruction. 

Management  Comments.  The  Acting  Director,  Defense  Contract  Management 
Agency  partially  concurred,  stating  that  DCMA  Raytheon  Integrated  Defense 
Systems  will  establish  an  LOD  at  Thales  Raytheon  Systems  to  support  the 
upcoming  software  qualification  testing.  He  stated  that  the  software  qualification 
testing  at  Thales  was  currently  delayed,  based  on  the  late  delivery  of 
Government-furnished  equipment  but  will  be  completed  before  the  end  of  the 
second  quarter  FY  2008.  He  further  stated  that  Raytheon  Network  Centric 
Systems  had  completed  work  and  that  any  future  delegations  would  be  limited  to 
rework  or  repair  of  the  items  delivered  to  Raytheon  Integrated  Defense  Systems. 
Therefore,  he  concluded  that  a  formal  delegation  for  Raytheon  Net  Centric 
Systems  was  not  necessary  or  cost  effective. 

Audit  Response.  The  acting  director’s  commitment  to  establish  an  LOD  at 
Thales  Raytheon  Systems,  the  remaining  active  subcontractor,  was  responsive  to 
the  recommendation. 
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D.  Implementing  Information  Assurance 
Guidance 

The  SLAMRAAM  Product  Office  changed  to  an  infonnation  assurance 
strategy  that  does  not  comply  with  issued  and  approved  DoD  information 
assurance  requirements.  Specifically,  the  product  office,  at  the  direction 
of  the  U.S.  Anny  Chief  Information  Officer,  began  following  the 
unapproved  DoD  Infonnation  Assurance  Certification  and  Accreditation 
Process  (DIACAP)  guidance  that  was  in  coordination  within  DoD,  but  not 
approved  for  implementation.  The  DoD  Infonnation  Technology  Security 
Certification  and  Accreditation  Process  (DITSCAP)  remains  the  official 
DoD  information  assurance  guidance.  This  occurred  because  the 
Assistant  Secretary  of  Defense  for  Networks  and  Information 
Integration/DoD  Chief  Information  Officer  issued  interim  information 
assurance  guidance,  DIACAP,  before  proper  coordination  and  review  by 
the  Director,  Administration  and  Management,  Office  of  the  Secretary  of 
Defense.  The  Army  Chief  Information  Officer  then  directed  the  product 
office  to  follow  this  interim  information  assurance  guidance.  Until  the 
DIACAP  guidance  is  properly  coordinated  and  reviewed,  the  product 
office  places  the  infonnation  contained  in  the  SLAMRAAM  system  at 
greater  risk  of  loss,  misuse,  or  unauthorized  access  to  or  modification  of 
the  information  contained  in  the  system. 


Information  Assurance  Policy 


DoD  Directive  5025.1.  DoD  Directive  5025.1,  “DoD  Directives  System,” 

July  14,  2004,  provides  policy  and  responsibilities  governing  DoD  directives, 
instructions,  and  publications.  The  directive  requires  the  Director,  Administration 
and  Management  to  review  and  coordinate  DoD  issuances. 

DoD  Instruction  5200.40.  DoD  Instruction  5200.40,  “DoD  Infonnation 
Technology  Security  Certification  and  Accreditation  Process  (DITSCAP),” 
December  30,  1997,  requires  the  heads  of  the  DoD  Components  to  implement  the 
DITSCAP  for  security  certification  and  accreditation  of  DoD  Components  and 
DoD  contractor  information  technology  systems  and  networks. 


Information  Assurance  Requirements 


While  the  SLAMRAAM  Product  Office  did  have  an  information  assurance 
strategy  and  a  contract  statement  of  work  that  specified  the  contractor  using 
DITSCAP  for  the  infonnation  assurance  requirements  as  mandated  in  DoD 
Instruction  5200.40,  the  SLAMRAAM  Product  Office  changed  this  strategy. 
SLAMRAAM  infonnation  assurance  personnel  stated  that  the  Army  Chief 
Information  Officer  directed  them  to  use  the  interim  DIACAP  guidance  to  certify 
and  accredit  the  infonnation  system  portion  of  the  SLAMRAAM  Program. 
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Properly  Coordinating  Interim  Guidance 


In  directing  the  SLAMRAAM  Product  Office  to  use  interim  DIACAP  guidance, 
the  Army  Chief  Information  Officer  reacted  to  an  interim  guidance  memorandum 
from  the  DoD  Chief  Information  Officer.  Specifically,  on  July  6,  2006,  the 
Assistant  Secretary  of  Defense  for  Networks  and  Infonnation  Integration/DoD 
Chief  Information  Officer  issued  the  memorandum,  “Interim  Department  of 
Defense  (DoD)  Infonnation  Assurance  (IA)  Certification  and  Accreditation 
(C&A)  Process  Guidance,”  that  established  interim  guidance,  including  a  draft 
instruction,  for  the  DIACAP.  Although  proper  coordination  and  review  by  the 
Director,  Administration  and  Management  had  not  occurred,  the  Assistant 
Secretary  stated  that  the  interim  guidance  superseded  the  DITSCAP  and 
established  procedures  for  infonnation  assurance  certification  and  accreditation  of 
DoD  information  systems  to  identify,  design,  test,  and  monitor  information 
assurance  capabilities  and  services. 

DoD  Directive  5025.1  requires  proposed  DoD  issuances  and  cancellations  to  be 
formally  coordinated  with  heads  of  DoD  Components  to  solicit  their  views.  In 
addition,  the  directive  requires  issuances  to  be  coordinated  with  the  DoD  General 
Counsel;  the  Inspector  General,  DoD;  and  reviewed  by  the  Director, 
Administration  and  Management.  On  August  2,  2006,  a  representative  for  the 
Washington  Headquarters  Services  sent  an  e-mail  to  the  Office  of  the  Assistant 
Secretary  of  Defense  for  Networks  and  Information  Integration  stating  that  the 
DIACAP  interim  guidance  could  not  supersede  the  DITSCAP  guidance  because 
the  Office  of  the  Assistant  Secretary  did  not  properly  coordinate  the  interim 
guidance  in  accordance  with  DoD  Directive  5025.1. 

As  part  of  the  coordination  process  under  DoD  Directive  5025.1,  the  DoD  Office 
of  Inspector  General  nonconcurred  with  the  April  2006  version  of  the  DIACAP 
interim  guidance  and  draft  instructions.  In  DoD  Inspector  General  memorandum, 
“Coordination  of  Draft  DoD  Instruction  8510.bb,  DoD  Information  Assurance 
Certification  and  Accreditation  Process  (DIACAP),”  May  25,  2007,  the  Principal 
Deputy  Inspector  General  listed  15  comments,  which  provided  the  basis  for  the 
nonconcurrance.  The  DoD  Directive  further  states  that  concerned  parties  should 
resolve  nonconcurrances  before  approving  and  publishing  an  issuance.  Key 
among  the  1 5  comments  that  the  DoD  Inspector  General  provided  were  that  the 
draft  instruction  did  not: 

•  include  specific  benchmarks  or  criteria  for  certifying  and  accrediting 
DoD  compliance  with  the  statutory  requirements  of  the  Chief 
Financial  Officer’s  Act  and  the  Federal  Information  Security 
Management  Act, 

•  include  DoD  contractors  and  agents  within  its  applicability  and  scope, 
nor 

•  establish  a  configuration  control  and  management  process  to  guide 
DoD  implementation  of  the  instruction. 
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According  to  a  representative  from  Washington  Headquarters  Services,  until  the 
DIACAP  interim  guidance  and  the  proposed  draft  instruction  are  coordinated  and 
reviewed  by  the  Director,  Administration  and  Management,  the  DITSCAP 
remains  the  official  DoD  infonnation  assurance  certification  and  accreditation 
policy. 


Effect  of  Improper  Information  Assurance  Guidance 


The  improperly  issued  interim  DoD  infonnation  assurance  guidance  caused  the 
Anny  Chief  Information  Officer  to  improperly  issue  his  own  direction  to  follow 
DIACAP  guidance  in  preparing  the  SLAMRAAM  Program  for  the  LRIP  decision 
program  review.  In  addition,  issuance  of  DIACAP  guidance  before  proper 
coordination,  review,  and  implementation  could  place  systems,  such  as 
SLAMRAAM,  at  a  greater  risk  of  loss,  misuse,  or  unauthorized  access  to  or 
modification  of  the  information  contained  in  the  systems. 

We  reported  a  similar  problem  with  premature  implementation  of  DIACAP  in 
DoD  Inspector  General  Report  No.  D2007-103,  “Air  Force  KC-X  Aerial 
Refueling  Tanker  Aircraft  Program,’’  May  30,  2007.  The  report  recommended 
that: 


The  Director,  Administration  and  Management,  according  to  his 
authority,  and  in  coordination  with  the  Assistant  Secretary  of  Defense 
for  Networks  and  Information  Integration/DoD  Chief  Information 
Officer  issue  formal  authoritative  information  assurance  policy  in 
accordance  with  DoD  Directive  5025.1,  “DoD  Directives  System,” 

July  14,  2004,  or  issue  guidance  to  the  Air  Force  regarding  applicable 
information  assurance  certification  and  accreditation  processes  for 
inclusion  in  its  KC-X  contractual  documents. 

The  Director,  Administration  and  Management  provided  comments  to  the  final 
report  stating  that  the  Director,  Administration  and  Management  does  not  have 
the  authority  to  issue  policy  or  guidance  on  information  assurance.  The  DoD 
official  with  the  responsibility  and  authority  to  issue  information  assurance  policy 
and  guidance  is  the  Assistant  Secretary  of  Defense  for  Networks  and  Infonnation 
Integration/DoD  Chief  Information  Officer.  Until  the  Assistant  Secretary 
properly  coordinates  new  DoD-wide  policy  and  guidance  that  cancels  DoD 
Instruction  5200.40  and  DoD  Manual  8510.1-M,  “Department  of  Defense 
Information  Technology  Security  Certification  and  Accreditation  Process 
(DITSCAP)  Application  Manual,”  July  31,  2000,  the  Director  stated  that 
DITSCAP  still  constitutes  the  official  DoD  information  assurance  policy. 


Conclusion 


To  ensure  consistent  infonnation  assurance  certification  and  accreditation 
guidance  within  DoD,  the  Director,  Administration  and  Management  must 
“review  and  coordinate  all  DoD  issuances.”  Further,  the  Assistant  Secretary  of 
Defense  of  Networks  and  Information  Integration/DoD  Chief  Information  Officer 
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must  submit  any  infonnation  assurance  guidance  that  has  the  effect  of  canceling 
DoD  issuances  to  the  Director,  Administration  and  Management  for  coordination 
and  review. 

Since  the  Director,  Administration  and  Management  must  coordinate  and  review 
any  official  DoD  information  assurance  certification  and  accreditation  policy  that 
has  the  effect  of  canceling  DoD  issuance,  the  SLAMRAAM  Product  Office 
should  not  follow  the  interim  DIACAP  guidance  instead  of  DITSCAP. 


Recommendations 


D.  1 .  We  recommend  that  the  Anny  Chief  Information  Officer  rescind  all  verbal 
or  written  direction  given  to  Army  acquisition  program  managers  to  follow  the 
interim  guidance  on  DoD  Information  Assurance  Certification  and  Accreditation 
Program  in  developing  weapon  systems. 

Management  Comments  Required.  The  Army  Chief  Information  Officer  did 
not  provide  comments  on  the  recommendation.  We  request  that  the  Anny  Chief 
Information  Officer  provide  comments  on  the  final  report. 

D.2.  We  recommend  that  the  Product  Manager,  Surface-Launched  Advanced 
Medium-Range  Air-to-Air  Missile  return  to  following  the  DoD  Infonnation 
Technology  Security  Certification  and  Accreditation  Process  in  developing  the 
Surface-Launched  Advanced  Medium-Range  Air-to-Air  Missile,  as  documented 
in  the  information  assurance  strategy  and  in  accordance  with  DoD  Instruction 
5200.40,  “DoD  Infonnation  Technology  Security  Certification  and  Accreditation 
Process  (DITSCAP),”  December  30,  1997. 

Management  Comments  Required.  The  product  manager  did  not  provide 
comments  on  the  recommendation.  We  request  that  the  product  manager  provide 
comments  on  the  final  report. 
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Appendix  A.  Scope  and  Methodology 


We  conducted  this  performance  audit  in  accordance  with  generally  accepted 
government  auditing  standards  from  October  2006  through  August  2007.  Those 
standards  require  that  we  plan  and  perform  the  audit  to  obtain  sufficient, 
appropriate  evidence  to  provide  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives.  We  believe  that  the  evidence  obtained 
provides  a  reasonable  basis  for  our  findings  and  conclusions  based  on  our  audit 
objectives. 

During  the  audit,  we  evaluated  whether  management  was  cost  effectively 
developing  and  readying  the  program  for  the  low-rate  initial  production  phase  of 
the  acquisition  process.  We  reviewed  requirements  and  capabilities,  testing, 
systems  engineering,  contracting,  acquisition  strategy,  and  funding  documents 
dated  from  May  1999  through  June  2007.  We  interviewed  staff  from  offices  of 
the  Director,  Operational  Test  and  Evaluation;  the  Army  Director  of  Combat 
Development;  the  Project  Manager,  Cruise  Missile  Defense  Systems;  the  Product 
Manager,  Surface-Launched  Advanced  Medium-Range  Air-to-Air  Missile;  the 
Commander,  Army  Test  and  Evaluation  Command;  and  the  Director,  Defense 
Contract  Management  Agency. 

Use  of  Computer-Processed  Data.  We  did  not  use  computer-processed  data  to 
perform  this  audit. 

Use  of  Technical  Assistance.  An  electrical  engineer  and  a  computer  engineer 
from  the  Electronics  Engineering  and  Information  Technology  Branches, 
Technical  Assessment  Directorate  of  Policy  and  Oversight,  Department  of 
Defense  Office  of  Inspector  General  assisted  in  the  audit.  The  engineers 
evaluated  and  reviewed  systems  engineering,  software,  and  other  acquisition 
planning-related  documents  in  the  SLAMRAAM  Program. 

Government  Accountability  Office  High-Risk  Area.  The  Government 
Accountability  Office  has  identified  several  high-risk  areas  in  DoD.  This  report 
provides  coverage  of  the  “DoD  Weapons  Systems  Acquisition”  high-risk  area. 


Prior  Coverage 


No  prior  coverage  has  been  conducted  on  the  overall  management  of  the 
SLAMRAAM  Program  during  the  last  5  years. 
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Appendix  B.  Measurable  and  Testable  Capability 

Requirements 


The  Chairman  of  the  Joint  Chiefs  of  Staff  (CJCS)  Manual  3170.01C  states  that 
the  capability  production  document  (CPD)  should  define  each  performance 
attribute  in  measurable  and  testable  terms.  However,  as  drafted,  the  CPD  did  not 
contain  measurable  and  testable  requirements  for  two  key  perfonnance 
parameters  and  six  other  performance  attributes  as  discussed  below. 


Key  Performance  Parameters 


Net-Ready.  CJCS  Instruction  62 12.0 ID  requires  CPDs  to  include  16  integrated 
architecture  products  that  provide  operational,  system,  and  technical  standard 
views  that  describe  the  system  and  its  interfaces  with  external  systems.  The  CJCS 
Instruction  states  that  the  descriptions  of  architecture  products  are  an  element  of 
the  net-ready  key  performance  parameter  and  that  sponsors  use  this  key 
performance  parameter  to  define  system  characteristics  and  key  perfonnance 
metrics  for  the  timely,  accurate,  and  complete  exchange  of  information. 

However,  the  draft  CPD  did  not  contain  architectural  product  descriptions  to 
support  the  net-ready  key  performance  parameter.  The  following  table  lists  the 
descriptions  of  the  architecture  products,  which  CJCS  Instruction  62 12.0  ID 
requires  program  sponsors  to  include  in  CPDs. 

Description  of  Architecture  Products 


Framework 

Products 

Framework  Product 
Name 

General  Description 

AV-1 

Overview  and  Summary 
Information 

Scope,  purpose,  intended  users,  environment  depicted,  and 
analytical  findings. 

OV-1 

High-Level  Operational 
Concept  Graphic 

High-level  graphical/textual  description  of  operational  concept. 

OV-2 

Operational  Node 
Connectivity  Description 

Operational  nodes,  operational  activities  performed  at  each  node, 
connectivity,  and  information  exchange  lines  between  nodes. 

OV-3 

Operational  Information 
Exchange  Matrix 

Information  exchanged  between  nodes  and  the  relevant  attributes 
of  that  exchange. 

OV-4 

Organizational 
Relationships  Chart 

Organizational  role  or  other  relationships  among  organizations. 

OV-5 

Operational  Activity 
Model 

Operational  activities,  relationships  among  activities,  inputs,  and 
outputs. 

OV-6c 

Operational  Event-Trace 
Description 

One  of  three  products  used  to  describe  operational  activity 
sequence  and  timing — traces  actions  in  a  scenario  or  sequence  of 
events  and  specifies  timing  of  events. 

OV-7 

Logical  Data  Model 

System  data  requirements  and  structural  business  process  rules  of 
the  operational  view. 

SV-1 

System  Interface 
Description 

Identification  of  systems  nodes,  systems,  and  system  items  and 
their  interconnections,  within  and  between  nodes. 
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Description  of  Architecture  Products  (Continued) 


SV-2 

System  Communications 
Description 

Systems  nodes  and  their  related  communications  lay-downs. 

SV-4 

System  Functionality 
Description 

Functions  performed  by  systems  and  the  information  flow  among 
system  functions,  including  information  assurance  functions. 

SV-5 

Operational  Activity  to 
Systems  Function 
Traceability  Matrix 

Mapping  of  systems  back  to  operational  capabilities  or  of  system 
functions  back  to  operational  activities. 

SV-6 

Systems  Data  Exchange 

Provides  details  of  systems  data  being  exchanged. 

SV-11 

Physical  Schema 

Physical  implementation  of  Logical  Data  Model  entities;  for 
example,  message  format,  file  structures,  and  physical  schema. 

TV-1 

Technical  Standards 
Profile 

Extraction  of  standards  that  apply  to  the  given  architecture, 
including  information  assurance  functions. 

TV-2 

Technical  Standards 
Forecast 

Emerging  standards  that  are  not  currently  approved.  The  TV-2 
should  also  be  used  to  document  technical  issues  affecting 
program  implementation. 

According  to  DCD  staff,  they  were  working  to  develop  the  above  architectural 
products  but  had  not  included  them  in  the  draft  CPD.  The  DCD  staff  stated  that 
they  planned  to  include  the  architectural  views  in  an  updated  draft  of  the  CPD. 


Other  Performance  Attributes 
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b(3) 


Appendix  C.  Audit  Response  to  Management 

Comments  on  the  Report 


Our  detailed  responses  to  the  comments  from  the  Project  Manager,  Cruise  Missile 
Defense  System  and  the  Acting  Director,  Defense  Contract  Management  Agency 
on  statements  in  the  draft  report  follow.  The  complete  text  of  the  management 
comments  is  in  the  Management  Comments  section  of  this  report. 


Project  Manager,  Cruise  Missile  Defense  Systems  Comments 
and  Audit  Response 


Management  Comments.  The  project  manager  provided  comments  on  the 
“Background”  section  of  the  report,  as  well  as  on  Findings  A,  B,  and  C. 

Background.  The  project  manager  stated  that  the  draft  report  incorrectly 
asserted  that  factors  causing  the  program  rebaselining  and  associated  cost  increase 
included  the  Army  not  sufficiently  defining  capability  requirements  in  the 
operational  requirements  document  used  to  develop  the  draft  capability 
production  document  (CPD),  contractor  difficulties  in  understanding  and 
allocating  system  requirements  to  subcontractors,  and  the  DCMA  Commander  not 
completely  adhering  to  policy  and  guidance.  The  project  manager  stated  that 
there  is  no  evidence  to  support  any  correlation  between  these  factors  and  the  need 
to  rebaseline  the  SLAMRAAM  Program.  The  project  manager  asserted  that  the 
rebaseline  occurred  because  of  the  following:  the  contractor  failed  to  follow 
established  engineering  processes,  Congress  denied  Army  requests  during  2006 
for  additional  funding  to  cover  contractor  overruns,  and  the  Marine  Corps 
withdrew  from  the  program  in  2006. 

Audit  Response.  We  revised  the  “Background”  section  to  state  that 
engineering  and  funding  problems,  along  with  the  Marine  Corps  withdrawing 
from  the  program,  were  the  primary  reasons  for  the  program  rebaseline. 

However,  we  also  believe  that  the  Army’s  shortfalls  in  defining  capability 
requirements  in  the  operational  requirements  document,  contractor  difficulties  in 
allocating  system  requirements  to  subcontractors,  and  the  DCMA  Commander  not 
adhering  to  policy  and  guidance  also  contributed  to  the  cost  and  schedule 
overruns  leading  to  the  rebaselining. 

Finding  A.  The  project  manager  provided  comments  on  the  finding 
paragraph  and  the  finding  section,  “System  Effectiveness  as  a  Key  Perfonnance 
Parameter.” 


Finding  Paragraph.  The  project  manager  disagreed  with  the 
statement  that  the  Director  of  Combat  Developments,  U.S.  Army  Air  Defense 
Artillery  School  did  not  sufficiently  define  capability  requirements  for  the 
SLAMRAAM  in  the  draft  CPD  being  prepared  for  Army  staffing  to  support  the 
LRIP  decision  program  review  planned  for  FY  2010.  The  project  manager  stated 
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that  the  July  2007  draft  CPD  that  the  audit  team  reviewed  required  additional 
information  and  was  not  meant  to  be  the  final  product.  He  explained  that  the 
Anny  finalizes  the  CPD  after  the  design  readiness  review,  validation,  and 
approval  before  the  LRIP  acquisition  decision.  He  further  stated  that  the  CJCS 
Manual  3170.01C,  “Operation  of  the  Joint  Capabilities  Integration  and 
Development  System,”  May  1,  2007,  does  not  require  system  effectiveness  to  be  a 
key  performance  parameter.  He  further  stated  that  the  CJCS  Manual  provides 
that,  because  a  CPD  is  finalized  after  the  design  readiness  review  and  after  the 
majority  of  capability  development,  it  is  normally  not  appropriate  to  introduce 
new  requirements  at  this  point. 

Audit  Response.  We  recognize  that  the  draft  CPD  was  not  a  final 
product  and  still  required  additional  information.  Therefore,  we  revised  the  report 
to  state  that  the  DCD  had  not  yet  (versus  “did  not”)  sufficiently  defined  capability 
requirements  in  the  draft  CPD.  With  regard  to  systems  effectiveness,  the  report 
does  not  state  that  system  effectiveness  is  required  as  a  key  performance 
requirement.  Rather  the  report  cites  the  CJCS  Manual  3 170.0 1C  definition  of  key 
performance  parameters  as  perfonnance  attributes  of  a  system  that  are  considered 
critical  or  essential  to  the  development  of  an  effective  military  capability  and 
make  a  significant  contribution  to  the  key  characteristics  of  the  program.  Also,  in 
elevating  system  effectiveness  to  a  key  performance  parameter,  we  are  not 
introducing  a  new  requirement.  We  are  simply  placing  additional  emphasis  and 
oversight  on  an  existing  requirement. 

System  Effectiveness  as  a  Key  Performance  Parameter.  The 

project  manager  disagreed  with  our  assertion  that  DCD  encountered  resistance 
from  the  SLAMRAAM  Product  Office  in  establishing  system  effectiveness  as  a 
key  performance  parameter.  He  stated  that  there  was  no  resistance  and  that  he 
recognizes  system  effectiveness  as  an  important  requirement  for  the  warfighter. 

He  further  stated  that  system  effectiveness  is  contained  in  the  system  perfonnance 
specification  for  the  SLAMRAAM  and  flows  down  to  all  elements  of  the 
SLAMRAAM  system.  The  project  manager’s  other  comments  on  system 
effectiveness  as  a  key  performance  parameter  are  discussed  in  the  management 
comments  section  to  Recommendation  A. 

Audit  Response.  During  the  audit,  DCD  staff  clearly  stated  that 
they  had  earlier  discussed  establishing  system  effectiveness  as  a  key  perfonnance 
parameter  with  the  SLAMRAAM  Product  Office  and  that  the  product  office  staff 
had  questioned  this  proposed  change.  For  tonal  purposes,  we  have  revised  the 
report  to  state  that  the  product  office  staff  questioned,  rather  than  resisted, 
establishing  system  effectiveness  as  a  key  performance  parameter. 

Finding  B.  The  project  manager  provided  comments  that  addressed  the 
finding  sections,  “Developing  a  Systems  Engineering  Plan,”  “Technical 
Reviews,”  “Technical  Baseline  Approach,”  and  “Systems  Engineering  Process.” 

Developing  a  Systems  Engineering  Plan.  The  project  manager 
disagreed  with  report  statements  concerning  the  requirement  to  have  a  SEP  and 
the  contractor  SEMP  not  fully  meeting  the  requirements  of  the  SEP. 
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Requirements  for  Having  a  SEP.  The  project  manager 
disagreed  with  stating  that  SLAMRAAM  Product  Office  personnel  did  not  have  a 
SEP  to  support  the  systems  engineering  process  for  the  SLAMRAAM  in 
accordance  with  USD(AT&L)  policy  and  guidance  that  was  issued  in  2004 
concerning  preparation  of  a  SEP  early  in  the  program  life  cycle.  The  project 
manager  stated  that  the  SLAMRAAM  program  did  not  have  a  completed  draft 
version  of  the  SEP  because  the  program  underwent  a  system  development  and 
demonstration  acquisition  decision  in  September  2003,  and  was  not  required  to 
follow  the  USD(AT&L)  memorandum,  “Policy  for  Systems  Engineering  in 
DoD,”  February  20,  2004.  In  addition,  the  project  manager  stated  that  the  LRIP 
decision  is  scheduled  for  early  FY  2010  and  a  completed  version  of  the  draft  SEP 
should  be  ready  for  review  by  March  2008. 

Audit  Response.  We  acknowledged  in  the  finding  that  a 
USD(AT&L)  policy  did  not  require  the  SLAMRAAM  Program  to  have  a 
completed  SEP  until  the  LRIP  decision  planned  for  early  FY  2010.  However,  the 
USD(AT&L)  memorandum,  “Implementing  Systems  Engineering  Plans  in  DoD  - 
Interim  Guidance,”  March  30,  2004,  recommended  that  acquisition  managers 
develop  a  SEP  early  in  the  program’s  life  cycle.  Therefore,  we  revised  the  report 
to  state  that  the  SLAMRAAM  Program  did  not  have  a  SEP  to  support  the  systems 
engineering  process  for  the  SLAMRAAM  in  accordance  with  USD(AT&L) 
guidance. 


Contractor  SEMP.  The  project  manager  disagreed  with 
reporting  that  the  SLAMRAAM  Product  Office  staff  was  using  the  contractor- 
developed  “Surface  Launched  AMRAAM  (SLAMRAAM)  Systems  Engineering 
Management  Plan  (SEMP),”  October  10,  2005,  which  did  not  fully  meet  the 
requirements  of  the  SEP  because  it  did  not  include  key  information  that  DoD 
systems  engineering  policy  and  guidance  requires  or  encourages  project  managers 
to  define.  He  stated  that  the  contractor-developed  SEMP,  released  in  April  2004, 
did  address  the  areas  contained  in  the  USD(AT&L)  memorandum,  “Policy  for 
Systems  Engineering  in  DoD,”  February  20,  2004,  including  the  systems 
engineering  processes  to  be  applied  to  the  program,  the  system’s  technical 
baseline  approach,  and  the  integration  of  systems  engineering  into  the  program’s 
integrated  product  teams.  Specifically,  he  stated  that  the  SEMP  contains  the 
program  description,  technical  baseline  planning  and  controls,  program  planning 
and  controls,  the  integrated  product  team  structure,  systems  engineering 
processes,  engineering  specialty  integration,  and  product  transition. 

Audit  Response.  As  discussed  below  under  “Technical 
Baseline  Approach,”  we  revised  the  report  to  delete  statements  that  the  SEMP  did 
include  the  systems  technical  baseline  approach.  As  discussed  subsequently 
under  “Systems  Engineering  Process,”  we  did  not  revise  our  position  on  the 
SEMP  not  fully  defining  the  systems  engineering  process. 

Technical  Reviews.  The  CMDS  Project  Manager  disagreed  with 
the  statement  that  the  SEMP  did  not  include  event-driven  technical  reviews. 
Specifically,  the  project  manager  stated  that  technical  reviews  were  specified  in 
the  SEMP  and  in  the  Integrated  Master  Schedule.  While  the  project  manager 
acknowledged  that  the  SEMP  did  not  state  entrance  and  success  criteria  for  the 
technical  reviews,  he  stated  that  the  following  documents,  referenced  in  the 
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SEMP,  did  detail  the  event-driven  technical  review  process  from  program  start  to 
closure:  Raytheon’s  Integrated  Defense  System  “Systems  Engineering  Practices 
and  Procedures,”  and  Raytheon’s  “Integrated  Product  Development  System.” 

The  project  manager  further  stated  that  the  technical  review  process  contains  a 
series  of  10  “gates”  containing  entry  and  exit  criteria  for  conducting  technical 
reviews,  with  gates  1  through  4  for  reviews  internal  to  Raytheon  and  gates 
5  through  10  for  Government  reviews.  In  addition,  the  project  manager  stated 
that  the  SLAMRAAM  Product  Manager  issued  the  Design  Review  Campaign 
Plan  for  Raytheon  to  use  to  address  the  design  review  process,  including  design 
review  entry  and  exit  criteria.  The  project  manager  stated  that  the  SLAMRAAM 
Product  Office  followed  Program  Executive  Office  Policy  04-36,  “Program 
Executive  Office  Management  Control  Checklist”  for  conducting  event-driven 
reviews  and  ensuring  that  the  contractor  met  review  entry  and  exit  criteria. 

Lastly,  the  product  manager  stated  that  our  report  implied  that  the  SLAMRAAM 
Product  Manager  did  not  tie  technical  reviews  to  event-driven  criteria.  As  proof 
to  the  contrary,  the  project  manager  cited  a  March  9,  2005,  letter  in  the  contract 
that  stated  that  Raytheon  had  not  met  entrance  criteria  for  a  preliminary  design 
review,  and  therefore  could  not  proceed  with  development  effort  until  it  met  these 
prerequisites. 

Audit  Response.  We  revised  the  report  to  clarify  that,  while  the 
SEMP  does  specify  the  following  technical  reviews:  system  requirements  review, 
system  design  review,  critical  design  review,  and  test  readiness  review,  it  did  not 
provide  entrance  and  exit  criteria  for  these  reviews.  We  also  revised  the  report  to 
emphasize  the  importance  of  including  entrance  and  exit  criteria  for  the  technical 
reviews  specified  in  the  SEMP,  rather  than  imply  that  the  product  manager  never 
tied  technical  reviews  to  event-driven  criteria.  We  did  not  revise  the  report  to 
state  that  the  project  manager  had  established  entry  and  exit  criteria  for 
SLAMRAAM  technical  reviews  through  the  documents  and  policies  he 
referenced  in  his  comments.  While  these  documents  and  policies  did  describe  the 
event-driven  technical  review  process,  they  did  not  provide  specific  entry  and  exit 
criteria  for  the  technical  reviews  included  in  the  SEMP.  The  “SLAMRAAM 
Program  Manager’s  Guidelines  for  Conducting  Design  Reviews,”  August  2005, 
which  documents  the  Design  Review  Campaign  Plan,  provides  an  example  of  the 
process  description.  This  document  states  that  the  program  manager  requires 
integrated  product  teams  to  develop  comprehensive  entry  and  exit  criteria  for  his 
approval  before  each  review,  but  does  not  include  any  specific  entry  and  exit 
criteria.  We  also  reviewed  the  following  five  additional  documents  the  project 
manager  provided  in  response  to  the  draft  report: 

•  “Part,  Material  and  Process  (PMP)  Plan,”  June  1,  2005; 

•  “Draft  Fire  Unit  CDR  Readiness  Review,”  August  19,  2005; 

•  “CDR  Program  Risk  Assessment  Checklist,”  August  1,  2004; 

•  “Integrated  Product  Development  System  -  IPDS  Brochure,”  (no  date); 
and 

•  “Independent  Review  8  (IR8)  Checklist  -  Detailed  Design  (Build 
Readiness),”  October  31,  2003. 


32 


With  the  exception  of  the  “Draft  Fire  Unit  CDR  Readiness  Review,”  August  19, 
2005,  which  contained  entry  and  exit  criteria  for  conducting  the  critical  design 
review  on  the  fire  unit,  none  of  the  above  documents  included  specific  entrance  or 
exit  criteria  for  planned  program  technical  reviews. 

Technical  Baseline  Approach.  The  CMDS  Project  Manager 
stated  that  the  SEMP  outlines  the  technical  baseline  approach  in  summary  fonn, 
while  the  technical  performance  baseline  is  contained  in  Government-controlled 
system  performance  specifications  and  various  contractor  specifications.  He 
stated  that  this  documentation  was  available  in  the  SLAMRAAM  Integrated  Data 
Environment  through  the  requirements  database  tools. 

Audit  Response.  We  revised  the  report  by  deleting  the  discussion 
of  the  technical  baseline  approach.  While  the  SEMP  did  not  specifically  mention 
technical  baseline  approach,  the  section  “Technical  Planning  and  Control”  did 
describe  procedures  for  establishing  a  baseline,  controlling  baseline  changes, 
tracing  system  requirements,  and  using  technical  perfonnance  measures  to 
monitor  and  control  the  program’s  perfonnance.  Therefore,  the  SEMP  did  meet 
the  intent  of  guidance  in  the  USD(AT&L)  memorandum,  “Implementing  Systems 
Engineering  Plans  in  DoD  -  Interim  Guidance,”  March  30,  2004,  for  defining  the 
technical  baseline  approach. 

Systems  Engineering  Process.  The  CMDS  Project  Manager 
disagreed  with  our  assertion  that  the  contractor-developed  SEMP  did  not  fully 
define  all  systems  engineering  processes  applied  to  the  SLAMRAAM.  He  stated 
that  the  SEMP,  together  with  Raytheon’s  “Systems  Engineering  Practices  and 
Procedures”  and  “Integrated  Product  Development  System,”  the  joint 
Raytheon/Govemment  “gate”  review  process  and  Design  Review  Campaign  Plan, 
Program  Executive  Officer  policies,  the  independent  assessment  team,  and  the 
Government-controlled  technical  baseline,  met  the  intent  of  the  SEP  guidance  and 
provided  him  with  a  well-defined  systems  engineering  process  and  technical 
baseline  approach  for  the  SLAMRAAM.  The  CMDS  Project  Manager  further 
stated  that  the  systems  engineering  processes  allowed  SLAMRAAM  to  recover 
from  a  major  contractor  performance  shortfall  to  a  highly  successful  critical 
design  review  some  1 5  months  later. 

Audit  Response.  We  did  not  revise  the  report  to  state  that 
documents  and  policies  the  project  manager  discusses  together  met  the  intent  of 
the  SEP  guidance  for  defining  the  systems  engineering  process.  The 
USD(AT&L)  memorandum,  “Implementing  Systems  Engineering  Plans  in  DoD  - 
Interim  Guidance,”  March  30,  2004,  states  that  the  SEP  should  be  a  living 
document  that  includes  updates  of  program  progress,  such  as  the  results  of 
completed  technical  reviews.  The  SEMP  had  not  been  updated  to  reflect  the 
systems  engineering  progress  since  October  2005. 

Finding  C.  The  project  manager  provided  comments  that  addressed  the  finding 
paragraph  and  the  finding  sections  “Establishing  Letters  of  Delegation,”  and 
“Conclusion.” 

Finding  Paragraph.  The  project  manager  disagreed  with  our  assertion 
that  the  Commander,  Defense  Contracting  Management  Agency  did  not  fully 


33 


TCT1 


describe  DCMA  activities  to  support  the  SLAMRAAM  Program  in  a  surveillance 
plan  and  establish  letters  of  delegation  with  other  DCMA  contract  management 
offices  for  surveillance  at  three  of  four  SLAMRAAM  subcontractors.  The  project 
manager  stated  that  DCMA  provided  excellent  support  and  kept  the 
SLAMRAAM  Product  Manager  informed  on  progress  regardless  of  the  content  or 
structure  of  the  MO  A  or  the  existence  of  LODs. 

Audit  Response.  We  did  not  revise  the  report  based  on  the  project 
manager’s  comments.  The  report  correctly  states  that  the  commander  did  not 
fully  describe  DCMA  activities  to  support  the  SLAMRAAM  Program  in  a 
surveillance  plan  and  establish  letters  of  delegation  with  other  DCMA  contract 
management  offices  for  surveillance  at  three  of  four  SLAMRAAM 
subcontractors.  While  the  project  manager  may  claim  satisfaction  with  DCMA 
support,  establishing  a  surveillance  plan  and  LODs,  as  discussed  in  the  finding, 
would  have  helped  DCMA  contract  management  offices  to  better  plan  and  track 
surveillance  support  activities.  Additionally,  the  DCMA  Instruction  requires  an 
outcome-oriented  LOD  when  the  prime  contractor  contracts  to  a  subcontractor  to 
work  on  specified  tasks. 

Establishing  Letters  of  Delegation.  The  project  manager  disagreed  that 
an  LOD  with  the  DCMA  Support  Program  Integrator  at  Boeing  Huntsville,  which 
required  fonnal  reports  and  earned  value  management,  would  have  provided  the 
project  manager  with  more  meaningful  infonnation  towards  satisfying 
SLAMRAAM  cost,  schedule,  and  performance  requirements.  He  stated  that  the 
SLAMRAAM  Program  received  cost,  schedule,  and  performance  information 
from  a  DCMA  Support  Program  Integrator  at  the  Boeing  Huntsville,  Alabama, 
plant  and  that  this  program  integrator  was  listed  in  Annex  D  of  the  MOA.  He 
additionally  stated  that  the  support  program  integrator  and  SLAMRAAM  Product 
Office  personnel  interfaced  on  a  nearly  daily  basis.  In  summary,  the  project 
manager  stated  that,  even  without  an  LOD  in  place,  the  SLAMRAAM  Product 
Manager  received  the  required  level  of  support  from  DCMA  on  the  Boeing 
subcontract  and  that  there  was  no  evidence  to  suggest  that  the  lack  of  an  LOD 
affected  the  quality  or  quantity  of  information  concerning  subcontractor  progress 
or  status. 

Audit  Response.  We  did  not  revise  the  report  based  on  the  project 
manager’s  comments.  The  lack  of  an  LOD  with  DCMA  Boeing  did  have  an 
impact  on  reporting  to  the  SLAMRAAM  Program.  Specifically,  we  spoke  to  the 
DCMA  Boeing  Program  Integrator  listed  in  Appendix  D  of  the  MOA  concerning 
the  impact  of  not  having  an  LOD  with  DCMA  Boeing.  He  stated  that  the 
SLAMRAAM  Program  Integrator  did  request  him  to  provide  earned  value 
management  support  on  Boeing’s  cost  and  schedule  progress,  but,  without  an 
approved  LOD,  he  was  not  able  to  assign  personnel  resources  to  support  earned 
value  reporting  to  the  SLAMRAAM  Program. 

Conclusion.  The  project  manager  disagreed  with  our  assertion  that, 
without  a  focused  and  comprehensive  MOA  supported  by  a  surveillance  plan  and 
LODs,  DCMA  staff  was  not  in  a  position  to  provide  the  Project  Manager,  CMDS 
with  timely  and  meaningful  insights  and  recommendations  regarding  the 
contractor  progress  toward  attaining  cost,  schedule,  and  performance  contract 
requirements  for  SLAMRAAM.  The  project  manager  stated  that  there  was  no 
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quantifiable  evidence  to  show  that  any  issues  with  the  MOA  affected  the  support 
provided  by  DCMA  or  the  quality  or  quantity  of  information  provided  concerning 
contractor  progress  and  status.  He  further  stated  that  the  quality  of  an  MOA  did 
not  necessarily  equate  to  quality  support,  nor  can  one  conclude  that  problems  with 
an  MOA  automatically  equate  to  problems  with  the  support. 

Audit  Response.  We  recognize  that  it  is  difficult  to  quantify  exactly  how 
much  better  DCMA  support  would  have  been,  had  there  been  a  focused  and 
comprehensive  MOA,  supported  by  a  surveillance  plan  and  LODs.  Therefore,  we 
revised  the  report  to  state  that  DCMA  staff  were  not  in  a  position  to  provide  the 
Project  Manager,  CMDS  with  the  most  timely  and  meaningful  insights  and 
recommendations  on  contractor  progress  toward  attaining  cost,  schedule,  and 
performance  contract  requirements  for  SLAMRAAM.  The  statements  of  the 
Boeing  program  integrator  under  “Establishing  Letters  of  Delegation”  above  did 
provide  one  example  of  a  specific  negative  impact  on  program  reporting  with 
regard  to  earned  value.  While  we  recognize  that  a  quality  MOA  does  not 
necessarily  equate  to  quality  support,  DCMA  officials  stated  that  establishing 
performance  metrics  and  standards  in  the  MOA  to  focus  and  measure  DCMA’s 
efforts  in  meeting  customer-desired  outcomes  enables  DCMA  to  more  efficiently 
use  limited  resources  to  support  program  managers. 


Acting  Director,  Defense  Contract  Management  Agency 
Comments  and  Audit  Response 


Management  Comments.  The  acting  director  provided  comments  on  the 
“Background”  section  of  the  report  and  on  Finding  C. 

Report  Background.  The  acting  director  stated  that  it  was  probably  an 
overstatement  to  state  that  the  DCMA  Commander’s  lack  of  adherence  to  policy 
was  one  of  the  primary  causes  of  the  program  rebaselining. 

Audit  Response.  As  stated  earlier,  we  revised  the  Report  Background  to 
state  that  engineering  and  funding  problems,  along  with  the  Marine  Corps 
withdrawing  from  the  program,  were  the  primary  causes  for  the  program 
rebaseline. 

Finding  C.  The  acting  director  provided  comments  that  addressed  the  finding 
section,  “Actions  to  Improve  the  Quality  of  Surveillance  Plans.”  Specifically,  the 
acting  director  suggested  that  we  delete  the  statement  that  DCMA  did  not 
establish  a  mandatory  requirement  for  contract  management  offices  to  use 
surveillance  plans  or  activity  annexes  in  the  interim  until  full  implementation  of 
the  Electronic  Data  Management  System. 

Audit  Response.  We  did  not  make  the  suggested  change  as  DCMA  did  not 
establish  a  mandatory  requirement  for  contract  management  offices  to  use 
surveillance  plans  or  activity  annexes  in  the  interim  until  full  implementation  of 
the  Electronic  Data  Management  System. 
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Appendix  D.  Report  Distribution 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
Director,  Acquisition  Resources  and  Analysis 
Director,  Defense  Procurement  and  Acquisition  Policy 
Under  Secretary  of  Defense  (Comp troller)/Chief  Financial  Officer 
Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 

Director,  Program  Analysis  and  Evaluation 


Joint  Staff 

Director,  Joint  Staff 


Department  of  the  Army 

Assistant  Secretary  of  the  Army  (Financial  Management  and  Comptroller) 

Auditor  General,  Department  of  the  Anny 

Director  of  Combat  Developments,  Army  Air  Defense  Artillery  School 
Project  Manager,  Cruise  Missile  Defense  System 

Product  Manager,  Surface-Launched  Advanced  Medium-Range  Air-to-Air  Missile 


Department  of  the  Navy 

Naval  Inspector  General 

Auditor  General,  Department  of  the  Navy 


Department  of  the  Air  Force 

Auditor  General,  Department  of  the  Air  Force 


Other  Defense  Organizations 

Director,  Defense  Contract  Management  Agency 

Non-Defense  Federal  Organization 

Office  of  Management  and  Budget 
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Congressional  Committees  and  Subcommittees,  Chairman  and 
Ranking  Minority  Member 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 

Senate  Committee  on  Homeland  Security  and  Governmental  Affairs 
House  Committee  on  Appropriations 

House  Subcommittee  on  Defense,  Committee  on  Appropriations 

House  Committee  on  Anned  Services 

House  Committee  on  Oversight  and  Government  Refonn 

House  Subcommittee  on  Government  Management,  Organization,  and  Procurement, 
Committee  on  Oversight  and  Government  Reform 
House  Subcommittee  on  National  Security  and  Foreign  Affairs, 

Committee  on  Oversight  and  Government  Reform 
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Defense  Contract  Management  Agency 
Comments 


DEFENSE  CONTRACT  MANAGEMENT  AGENCY 

63SC  WALK6R  LANE,  SUITE  300 
ALEXANDRIA,  VA  22310-3J41 


IN  REPLY 
RereR  TO 


DCMA-DM1 


pens  2007 


memorandum  for  program  director,  acquisition  and  contract 

MANAGEMENT,  DEPARTMENT  OF  DEFENSE 
INSPECTOR  GENERAL,  OFFICE  OF  THE  DEPUTY 
INSPECTOR  GENERAL 


SUBJECT:  Department  of  Defense  -  Inspector  General  Draft  Report,  Project  Numba 
D2007-  DOOOAE-OO6O,  Report  on  the  Acquisition  of  Surface-Launched 
Advanced  Medium-Range  Air-to-Air  Missile 


Reference:  DoDIG  draft  audit  report,  Project  Number  D2OO7-DQ00AE-006O 
subject  as  above. 


We  have  attached  the  Headquarters  Defense  Contract  Management  Agency 
response  and  technical  comments  to  the  findings  and  recommendations  cited  in  the 
subject  report. 


Point  of  contact  is  | 
lnidcma.mil. 


| at  (70?)  530-^Bm 


ATTACHMENT 

As 
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SUBJECT:  Acquisition  of  the  Surface-Launched  Advanced  Medium-Range  Air-to-Air 
Missile  (Project  Number  D2OQ7-D0OOAE-OO6O) 

FINDING  C:  Establishing  Defense  Contract  Management  Agency  Support 
Responsibilities 

The  Commander,  Defense  Contract  Management  Agency  (DCMA)  Space 
Sensors  and  Communications  Operations,  Raytheon  Integrated  Defense  Systems  and 
the  Project  Manager,  Cruise  Missile  Defense  System  (CMDS)  did  not  adequately 
establish  and  document  needed  DCMA  support  in  the  memorandum  of  agreement 
(MOA)  for  the  SLAMRAAM  program.  Specifically,  the  MOA  did  not  reference  current 
DCMA  policy  and  fully  establish  links  between  the  MOA  performance  metrics  and  the 
project  manager’s  desired  outcomes  for  the  SLAMRAAM  program.  Additionally,  the 
commander  did  not  fully  describe  DCMA  activities  to  support  the  SLAMRAAM  program 
In  a  surveillance  plan  and  establish  letters  of  delegation  (LOD)  with  other  DCMA 
contract  management  offices  for  surveillance  at  three  of  four  SLAMRAAM 
subcontractors.  These  conditions  occurred  because  the  commander  did  not  completely 
adhere  to  provisions  in  the  Federal  Acquisition  Regulation,  the  DCMA  Instruction,  and 
the  DCMA  Guidebook  for  preparing  an  MOA,  formulating  a  surveillance  plan,  and 
establishing  LODs.  As  a  result,  DCMA  did  not  have  information  needed  to  provide  the 
project  manager  with  informed  recommendations  regarding  contractor  progress  toward 
attaining  contract  cost,  schedule,  and  performance  requirements. 

RECOMMENDATION  Cla  &  b 

We  recommend  that  the  Commander,  Defense  Contract  Management 
Agency  Space  Sensors  and  Communications  Operations,  Raytheon  Integrated  Defense 
Systems  and  the  Project  Manager,  Cruise  Missile  Defense  Systems  revise  the 
memorandum  of  agreement  for  the  Surface-Launched  Advanced  Medium-Range  Air-to- 
Air  Missile  to: 

a.  Reference  the  current  Defense  Contract  Management  Agency 
Instruction  and  Defense  Contract  Management  Agency  Guidebook. 

b.  Include  an  annex  that  documents  a  cause-and-effect  analysis  linking 
Defense  Contract  Management  Agency  performance  metrics  and 
standards  to  the  customer-desired  outcome,  in  accordance  with  the 
Defense  Contract  Management  Agency  Instruction. 

DCMA  COMMENTS:  Concur.  Concur  with  the  recommendations  as  stated.  DCMA 
IDS  is  in  process  of  updating  the  MOA  in  accordance  with  the  current  agency  guidelines 
and  instructions.  Target  date  for  completion  of  this  task  (MOA  update)  is  First  Quarter 
FY08. 

RECOMMENDATION  C2 


We  recommend  that  the  Acting  Director,  Defense  Contract  Management  Agency  update 
the  DCMA  Instruction  to  require  that  memorandums  of  agreement  with  acquisition 
program  managers  include  surveillance  plans  or  activity  annexes  that  prioritize  Defense 
Contract  Management  Agency  support  that  will  be  provided  to  an  acquisition  program, 
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SUBJECT:  Acquisition  of  the  Surface-Launched  Advanced  Medium-Range  Air-to-Air 
Missile  (Project  Number  D2007-DOOOAE-0060) 

performance  metrics  and  standards,  and  identify  what,  when,  where,  and  how  the 
Defense  Contract  Management  Agency  staff  will  perform  activities  to  support 
surveillance  until  the  electronic  Data  Management  System  is  completed, 

DCMA  COMMENTS:  Nonconcur.  DCMA  does  not  concur  with  the  recommendation. 
We  have  a  method  of  implementation  already  in  place  that  we  believe  is  more  effective 
than  requiring  that  the  noted  information  be  a  mandatory  part  of  the  MOA. 

DCMA  policy  does  not  require  the  inclusion  of  the  performance  commitment  strategy 
(surveillance  plan)  in  the  MOAs  we  execute  with  our  customers.  Inclusion  of  the 
strategy  (Annex  E  of  the  MOA)  is  discretionary  unless  requested  by  the  customer.  The 
reason  that  we  do  not  make  the  inclusion  of  Annex  E  mandatory  is  that  our  commitment 
is  not  to  the  activities  that  are  the  strategy,  but  to  the  results  (performance 
commitments)  noted  in  the  body  of  the  MOA.  This  distinction  is  the  very  heart  of  DCMA 
performance-based  management.  If  we  made  the  inclusion  of  Annex  E  mandatory, 
then  we  would  be  committing  not  only  to  the  results  desired,  but  to  a  strategy  (set  of 
activities)  that  may  need  to  change  if  the  analysis  shows  that  progress  toward  the 
results  was  not  satisfactory.  The  preference  for  results  Is  both  logical  and  consistent 
with  the  intent  of  the  Budget  and  Performance  Integration  direction  in  the  President's 
Management  Agenda,  the  Lean/Six  Sigma  initiative  of  the  USD  (AT&L)  and  other  DoD 
initiatives  to  reorient  the  management  of  the  Department  toward  performance  results. 
While  the  inclusion  of  Annex  E  is  not  mandatory  in  the  MOA,  DCMA  does  require  that 
contract  management  offices  execute  the  strategies  (aka  surveillance  plans)  to  achieve 
their  performance  commitments.  We  chose  another  way  of  ensuring  that  the  activities 
that  make  up  the  strategies  are  executed.  The  activities  that  comprise  the  strategies  are 
contained  in  the  Individual  Performance  Plans  (IPP)  of  the  functional  specialists.  By 
documenting  the  strategies  In  this  fashion,  we  make  execution  a  direct  linkage  between 
the  organizational  performance  commitments  and  the  factors  that  determine 
performance  of  the  DCMA  workforce.  To  ensure  that  the  strategies  are  always 
maximally  effective  we  have  made  the  organizational  performance  commitments  critical 
job  elements  of  the  IPP  of  the  individuals  who  supervise  those  executing  the  activities  of 
the  strategies,  This  linkage  of  organizational  and  individual  performance  makes  our 
PBM  approach  one  that  is  focused  on  appropriate  pnorities  and  "self-governing.'-  What 
needs  to  be  achieved  (the  results)  and  what  needs  to  be  done  to  achieve  those  results 
is  ingrained  In  our  normal  course  of  doing  business  and  in  the  rating  scheme  for  both 
our  organizational  and  individual  performance  assessments. 


RECOMMENDATION  C3 

We  recommend  that  the  Commander,  Defense  Contract  Management 
Agency  Space  Sensors  and  Communications  Operations,  Raytheon  Integrated  Defense 
Systems  establish  letters  of  delegation  with  supporting  Defense  Contract  Management 
Agency  contract  management  offices  for  subcontractor  surveillance  at  Raytheon 
Network  Centric  Systems  and  Thales  Raytheon  Systems  subcontractors,  In  accordance 
with  the  Defense  Contract  Management  Agency  Instruction. 
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SUBJECT:  Acquisition  of  the  Surface-Launched  Advanced  Medium-Range  Alr-to-Air 
Missile  (Project  Number  D2007-DOOOAE-0060) 

DCMA  COMMENTS:  Partially  Concur.  DCMA  IDS  will  establish  a  letter  of  delegation 
at  Thales  Raytheon  Systems  (TRS)  to  support  the  upcoming  Software  Qualification 
Testing  (SQT).  This  testing  is  currently  delayed  based  on  the  late  delivery  of 
Government  Furnished  Equipment  (GFE)  but  will  be  completed  prior  to  the  end  of  2nd 
quarter  FY  08. 

At  this  time  the  work  at  Raytheon  Network  Centric  Systems  (NCS)  is  completed  and  any 
future  delegations  would  be  limited  to  rework  or  repair  of  the  items  delivered  to 
Raytheon  IDS.  Therefore,  given  the  completion  of  the  work  at  Raytheon  NCS,  we 
consider  a  formal  delegation  to  not  be  necessary  or  cost  effective. 
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SUBJECT:  Acquisition  of  the  Surface-Launched  Advanced  Medium-Range  Air-to-Air 
Missile  (Project  Number  D2007-DOOOAE-0060) 

Technical  Comments; 

RECOMMENDED  CHANGES  TO  THE  REPORT 
STATEMENT: 

Factors  causing  the  rebaselining  and  associated  cost  Increase  included  the  Army  not 
sufficiently  defining  capability  requirements  in  the  operational  requirements  document 
used  to  develop  the  draft  capability  production  document  (CPD),  contractor  difficulties  in 
understanding  and  allocating  system  requirements  to 

subcontractors,  and  the  DCMA  commander  not  completely  adhering  to  policy  and 
guidance. 

Section:  Background  (Page  2) 

Paragraph:  Program  Rebaseline 

SUGGESTED  CHANGE: 

Factors  causing  the  rebaselining  and  associated  cost  increase  included  the  Army  not 
sufficiently  defining  capability  requirements  in  the  operational  requirements  document 
used  to  develop  the  draft  capability  production  document  (CPD)  and  contractor 
difficulties  in  understanding  and  allocating  system  requirements  to  subcontractors. 

A  factor  that  may  have  contributed  to  the  rebaselining  and  associated  cost  increase  was 
the  DCMA  commander  not  completely  adhering  to  policy  and  guidance. 

DCMA  COMMENTS: 

While  the  lack  of  adherence  to  policy  may  have  limited  the  visibility  of  the  SLAMRAAM 
Project  Manager  into  selected  cost  and  schedule  issues,  it  being  characterized  as  one 
of  the  primary  causes  of  the  rebaselining  is  probably  an  overstatement,  Recommend 
the  change  noted  above. 

STATEMENT: 

The  DCMA  officials  stated  that  performance-based  management  enables  DCMA  to 
more  efficiently  use  limited  resources  to  support  program  managers  by  establishing 
performance  metrics  and  standards  in  the  MOA  to  focus  and  measure  DCMA's  efforts  in 
meeting  customer-desired  outcomes. 

Section:  Defense  Contract  Management  Agency  Support  to  SLAMRAAM  (Page  12) 
Paragraph:  Applicable  Policy  and  Guidance 

SUGGESTED  CHANGE: 

Change  the  clause "...  DCMA's  efforts  in  meeting  customer-desired  outcomes.'1  in 
the  3™  sentence  (shown  above)  to "...  DCMA's  efforts  In  helping  customers  meet  their 
desired  outcomes.” 

STATEMENT: 

As  a  result,  DCMA  could  better  suoport  the  program  manager  by  focusing  its  limited 
resources  through  performance  metrics  and  standards  to  achieve  customer-desired 
outcomes  and  enable  him  to  provide  Informed  recommendations  on  contractor 
performance. 
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Revised 


Page  15 


Revised 


Page  15 


Not  deleted, 
see  Audit 
Response  to 
Management 
Comments 
to  Recom¬ 
mendation 
C.2  on  page 
18 


SUBJECT;  Acquisition  of  the  Surface-Launched  Advanced  Medium-Range  Air-to-AIr 
Missile  (Project  Number  D2007-D000AE-0060) 

Section:  Defense  Contract  Management  Agency  Support  to  SLAMRAAM  (Page  12) 
Paragraph:  Cause-and-Effect  Analysis  Annex 

SUGGESTED  CHANGE: 

In  the  3rd  sentence  (shown  above),  change  the  clause  ”...  to  achieve  customer-desired 
outcomes  and  enable  him  to  provide  Informed  recommendations  on  contractor 
performance "  to  “ ...  to  help  the  customer  achieve  its  desired  outcomes  and  enable  the 
program  manager  to  provide  informed  recommendations  on  contractor  performance." 

STATEMENT: 

Section:  Defense  Contract  Management  Agency  Support  to  SLAMRAAM  (Page  13) 
Paragraph:  Actions  to  Improve  the  Quality  of  Surveillance  Plans 

SUGGESTED  CHANGE: 

•  Capitalize  the  word  “electronic'’  in  the  lsl  sentence. 

•  Change  the  word  "provide"  in  the  1st  sentence  to  "link." 

DCMA  COMMENTS: 

•  The  application  is  the  Electronic  Data  Management  System  (EDMS). 

•  The  EDMS  will  not  “provide”  or  house  all  of  the  information  noted  but  will  be  the 
link  that  connects  that  information  and  enables  effective  configuration  control  of 
the  samo. 

STATEMENT: 

(3rd  paragraph;  2nd  sentence) ....  DCMA  did  not  establish  a  mandatory  requirement  for 
contract  management  offices  to  use  surveillance  plans  or  activity  annexes  in  the  interim. 
Section:  Defense  Contract  Management  Agency  Support  to  SLAMRAAM  (Page  13) 
Paragraph:  Actions  to  Improve  the  Quality  of  Surveillance  Plans 

SUGGESTED  CHANGE: 

Delete  paragraph  3. 

DCMA  COMMENTS: 

See  DCMA  response  to  finding  C.2  wherein  we  non  concur  with  the  recommendation  of 
adding  a  surveillance  plan  to  Annex  E  of  the  MOA. 
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EXECUTIVE  SUMMARY 

Comments  to  the  proposed  l)oI)  Inspector  General  Report  “Acquisition 
of  the  Surface-Launched  Advanced  Medium  Range  Air-to-Air  Missile” 

Project  No.  D2007-D000.4E-0060.000,  dated  August  17,  2007 

1 0  September  2007 


1 .  Overview 

The  Cruise  Missile  Defense  System  (CMDS)  Project  Office,  Product  Manager  (PM)  for 
the  Surface  Launched  Advanced  Medium  Range  Air-to-Air  Missile  (SLAMRAAM) 
System  has  reviewed  the  subject  document  and  offers  the  following  comments  to  the 
findings  stated  in  the  report. 

The  following  comments  address  statements  made  in  the  1G  Report  (in  italics)  followed 
by  the  PM-SLAMRAAM  rational  for  non-concurrence.  Specific  comments  to  the 
executive  summary  were  not  made  since  it  is  a  summary  of  the  findings. 

2.  IG  Report  -  Background  Section 

" Factors  causing  the  rebaselining  and  associated  cost  increase  included  the  Army  not 
sufficiently  defining  capability  requirements  in  the  operational  requirements  document 
used  to  develop  the  draft  capability  production  document  (CPD),  contractor  difficulties 
in  understanding  and  allocating  system  requirements  to  subcontractors,  and  the  DCMA 
commander  nut  completely  adhering  to  policy  and  guidance. "  (IG  Report  page  2) 

The  CMDS  Project  Office  non-concurs  with  this  conclusion. 

There  is  no  evidence  to  support  that  there  is  any  correlation  between  Ihe  need  to  re¬ 
baseline  the  SLAMRAAM  program  and  the  capability  requirements,  systems  engineering 
planning,  or  the  use  of  DCMA.  The  reasons  for  the  re-baseline  are  twofold:  1)  the 
contractor  failed  to  follow  established  engineering  processes  which  required  the 
Government  to  direct  that  the  contractor  (Raytheon  Integrated  Defense  Systems)  stop 
development  and  address  the  associated  problems,  and  2)  Congressional  action  in  2006 
that  denied  requests  for  additional  money  to  cover  the  contractor  overrun  and  the  Marine 
Corps  withdrawal  from  the  program  in  2006. 

The  contractor  performance  shortfall  was  documented  in  a  contracts  letter  to  the 
contractor  dated  March  9, 2005  (US  Army  Aviation  and  Missile  Command  AMSAM- 
AC-SM-T  (SL-007-05)).  The  letter  documented  the  contractor’s  "failure  to  execute  a 
rigorous  system  engineering  process,  delinquent  or  incomplete  data  deliveries, 
incomplete  traceability  of  requirements  through  the  hardware  and  software,  and  what 
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appeal's  to  be  a  fundamental  misinterpretation  of  some  contract  requirements  resulting  in 
unresolved  issues  with  respect  to  the  design  approaches,” 

The  letter  informed  Raytheon  that  the  Government  was  dissatisfied  with  its  manner  of 
performance  with  respect  to  delivery  of  system  engineering  products,  that  it  did  not  fulfill 
the  entrance  criteria  for  a  SLAMRAAM  System  Preliminary  Design  Review,  that  it 
would  not  proceed  with  software  development  until  authorized  by  the  Government,  and 
that  hardware  manufacturing  would  not  begin  until  the  requisite  design  reviews  are 
successfully  completed.  The  Government  requested  a  recovery  plan  which  included  a 
series  of  incremental  reviews  leading  to  a  System  Critical  Design  Review;  this  recovery 
plan  was  delivered  by  Raytheon  in  April  2005. 

The  Government  letter  documented  evidence  of  these  facts;  “the  Government  and 
Raytheon  have  jointly  developed  and  agreed  to  SPDR  (System  Preliminary  Design 
Review)  entry  criteria  that  further  specify  products  needed  to  hold  a  successful  review.  In 
the  Government’s  opinion  the  data  and  products  delivered  to  date  fail  to  meet  the 
requirements  . . .  Important  products  required  by  the  contract  or  entry  criteria  are 
unavailable  for  review  or  were  delivered  incomplete.  The  technical  adequacy  of  the 
derived  requirements  and  allocated  baseline  flow  down  is  deficient.  Softw'are 
requirements  are  not  complete  or  stable.  The  interfaces  among  the  Configuration  Items 
are  incomplete  and  deficient.  The  full  bi-directional  traceability  for  each  individual 
software  requirement  as  required  by  the  contract  and  the  contractual  Software 
Development  Plan  is  absent,  negating  the  ability  to  determine  the  maturity  of  the  software 
and  safety  requirements". 

On  Finding  B,  the  IG  repotted  the  following: 

"Without  tying  technical  reviews  to  event-driven  criteria  through  established  entrance 
and  success  criteria  for  these  reviews,  the  product  manager  cannot  ensure  that  technical 
reviews  are  held  only  when  program  accomplishments  make  them  warranted  and 
meaningful  in  measuring  program  results  (IG  Report  page  8).  ” 

This  statement  implies  that  the  PM  did  not  tic  technical  reviews  to  event-driven  criteria. 
As  can  be  seen  from  the  contract  letter  just  quoted,  that  was  not  the  case.  The  CMOS 
Project  Office  non  concurs  with  this  implication. 

A  strong  system  engineering  process  was  in  place;  the  contractor  failed  to  follow  it  and 
the  SLAMRAAM  Government  PM  used  effective  management  controls  to  correct  the 
situation. 

The  need  to  re-baseline  the  SLAMRAAM  program  was  not  due  to  the  capability 
requirements,  systems  engineering  planning,  or  the  use  of  DCMA.  These  areas  are 
discussed  in  the  Findings  below. 
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3.  Finding  A:  Defining  Capability  Requirements 

The  CMOS  Project  Office  non-concurs  with  this  finding. 

"The  Director  of  Combat  Developments  (DCD),  U.S.  Army  Air  Defense  Artillery  School, 
did  not  sufficiently  define  capability  requirements  for  the  SLAMRAAM  in  the  draft  CPD 
being  prepared  for  Army  staffing  to  support  the  LRIP  decision  program  review  planned 
for  FY  2010. "  (IG  Report  page  4 ) 

The  CMDS  Project  Office  non-concurs  with  this  statement. 

“A  CPD  is  finalized  after  design  readiness  review  and  is  validated  and  approved  before 
the  Milestone  C  acquisition  decision  (CJCSM  3170.01C)”.  The  SLAMRAAM  CPD  is 
currently  in  draft  form  and  requires  additional  information  to  support  the  Milestone  C 
decision  to  proceed  with  LRIP.  SLAMRAAM  has  not  yet  fully  transitioned  from  Systems 
Integration  to  System  Demonstration  (as  defined  in  DODI  5000.2)  as  the  system  has  yet 
to  perform  “adequate  development  testing”  as  required  by  DODI  5000.2.  This  testing  will 
mature  in  FY08.  During  FY08,  the  CPD  will  continue  to  mature  to  capture  not  only 
design  review  data  but  also  test  data  as  required.  The  current  draft  CPD  (July  2007) 
reviewed  by  the  IG  team  was  not  meant  to  be  the  final  product. 

"Specifically,  the  draft  CPD  did  not  identify  system  effectiveness  as  a  key  performance 
pa  rameter  ( IG  Report  page  4 )" 

The  CMDS  Project  Office  non  concurs  that  system  effectiveness  is  required  to  be  a  KPP. 

"The  CPD  refines  the  threshold  and  objective  values  for  performance  attributes  and  KPPs 
that  were  validated  in  the  CDD  for  the  production  increment  (CJCSM  3 1 70.0 1 C)”.  The 
SLAMRAAM  Milestone  B  was  supported  by  a  validated  Operational  Requirements 
Document  (ORD);  a  CDD  was  not  developed.  The  ORD  did  not  contain  system 
effectiveness  as  a  KPP.  The  current  CJCSM  3 170.0IC  (1  May  2007)  docs  not  require  that 
system  effectiveness  be  a  KPP.  The  CJCSM  guidance  states  that  “Because  a  CPD  is 
finalized  after  design  readiness  review  and  after  the  majority  of  capability  development, 
it  is  normally  not  appropriate  to  introduce  new  requirements  at  this  point  ” 

“During  the  audit,  DCD  staff  stated  that  they  intended  for  system  effectiveness  to  be  a 
key  peiformance  parameter  but  encountered  resistance  from  the  SLAMRAAM  Product 
Office.  DCD  staff  stated  that  the  SLAMRAAM  Product  Office  resisted  because 
SLAMRAAM  system  effectiveness  was  directly  related  to  proven  capabilities  of  the 
Government-furnished  AMRAAM,  which  the  product  office  had  no  influence  over  (IG 
Report  page  5)." 

The  CMDS  Project  Office  non-concurs  with  this  statement. 

There  is  no  “resistance”.  The  system  effectiveness  is  an  important  requirement  for  the 
war-fighter  and  an  important  performance  requirement,  just  not  a  KPP.  The  system 
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effectiveness  is  contained  in  the  SLAMRAAM  System  Performance  Specification  and 
has  been  flowed  down  to  all  elements  in  the  SLAMRAAM  System. 


System  effectiveness  was  not  a  KPP  at  the  Milestone  B  decision  because  SLAMRAAM 
relied  on  an  existing  missile  and  existing  sensots.  These  elements  contribute  heavily  to 
the  system  effectiveness  equation  but  the  performance  of  these  elements  is  beyond  the 
control  of  the  SLAMRAAM  Materiel  Developer.  This  was  correctly  noted  in  the  IG 
Report. 


The  parameters  that  are  under  the  control  of  the  SLAMRAAM  Materiel  Developer,  m 

2m89MSB3CMQBX8M^8BI9RSRi^H^0  contribute  to  the  system 
effectiveness.  But  making  them  KPPs  would  be  of  marginal  value  since  these  parameters 
each  may  be  met,  and  yet  the  overall  system  may  still  fall  short  of  a  desired  capability 
since  the  performance  of  the  missile  and  the  sensor  within  a  SLAMRAAM  System  has 
no^eerMlemonstrate^Likewise^hortfalls  may  be  identified  in 

the  overall  system  could  still  deliver  a  desired 
capability.  Predictions  indicate  a  viable  military  capability  but  the  system  effectiveness 
value  can  only  be  determined  at  the  end  of  the  SDD  phase,  where  the  true  performance  of 
the  missile  and  the  sensor  in  the  SLAMRAAM  System  context  can  be  determined. 


Thus,  the  system  effectiveness  remaias  a  performance  attribute  that  will  be  quantified  at 
Milestone  C  as  end-to-end  performance,  including  the  sensor  and  missile  performance.  A 
decision  can  be  made  at  that  time  to  assess  the  combat  utility  of  the  system. 


“Recommendation  A:  We  recommend  that  the  Directorate  of  Combat  Development,  CJ.S. 
A  rmy  Air  Defense  Artillery  School  revise  the  draft  capability  production  document  for  the 
Sutf ace -Launched  Advanced  Medium-Range  Air-to-Air  Missile  in  accordance  with  policy 
in  Chairman  Joint  Chiefs  of  Staff  Instruction  3 170.01  F,  “Joint  Capabilities  Integration 
and  Development  System,  "  May  I.  2007,  to  define  the  probability 
BHPBBBMHBiu  a  key  performance  parameter;  and  measurable 
and  testable  capability  requirements  I 


As  discussed  above,  the  CMOS  Project  Office  non-concurs  that  EflHlBHi 
MBHHHBinccd  to  be  KPPs.  The  CMOS  Project  Office 
concurs  that  all  CPD  capabilities  should  be  “measurable  and  testable".  The  current  draft 
CPD  (July  2007)  reviewed  by  the  IG  team  was  not  meant  to  be  the  final  product.  The 
SLAMRAAM  Product  Office  is  working  with  IK'D  to  ensure  that  the  CPD  meets 
established  CJCS  policy  for  defining  system  capability.  The  final  CPD  will  ensure  that 
the  SLAMRAAM  satisfies  essential  war-fighter  needs. 
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4.  Finding  B  -  Planning  for  Systems  Engineering 

Tiie  CMDS  Project  Office  non-concurs  with  this  finding. 

"SLAMRAAM  Product  Office  personnel  stated  they  did  not  have  a  SEP  to  support  the 
systems  engineering  process  for  the  SLAMRAAM  in  accordance  with  USD  (AT&L)  policy 
and  guidance  that  was  issued  in  2004  concerning  preparation  of  a  SEP  early  in  the 
program  life  cycle.  The  SLAMRAAM  Product  Office  staff  stated  that  while  they  had 
started  working  towards  a  SEP,  they  did  not  have  a  draft  version  of  the  SEP  for  us  to 
review  (IG  Report  page  8). " 

The  CMDS  Project  Office  non-concurs  with  this  statement. 

The  USD  (AT&L)  policy  states  that  “Programs  shall  develop  a  Systems  Engineering  Plan 
(SEP)  for  Milestone  Decision  Authority  (MDA)  approval  in  conjunction  with  each 
Milestone  review...”  (USD  (AT&L)  Memorandum,  “Policy  for  Systems  Engineering  in 
DoD,"  February  20. 2004).  The  SLAMRAAM  Program  underwent  a  Milestone  B  review 
in  September  2003;  this  policy  was  not  published  until  February  2004.  The  System 
Engineering  Plan  will  be  developed  to  support  the  Milestone  C  decision  in  early  FY10. 
That  decision  is  two  years  off  thus  a  complete  draft  was  not  available  for  review.  The 
SEP  is  currently  being  drafted  and  an  initial  draft  for  review  should  be  available  by 
March  2008. 

In  the  meantime,  the  SLAMRAAM  Product  Office  staff  was  using  the  contractor- 
developed  " Surfaced  Launched  AMRAAM  (SLAMRAAM)  Systems  Engineering 
Management  Plan  (SEMP), "  October  10,  2005.  to  support  the  systems  engineering 
process  for  the  program.  This  SEMP  did  not  fully  meet  the  requirements  of  the  SEP 
because  it  did  not  include  key  information  that  DoD  systems  engineering  policy  and 
guidance  requires  or  encourages  project  managers  to  define  (IG  Report  page  8).  ” 

The  CMDS  Project  Office  non-concurs  with  this  statement. 

The  contract  specified  that  the  contractor  prepare  a  SEMP  as  a  data  item  deliverable  to 
the  Government.  The  initial  version  was  released  in  April  2004,  two  months  after 
contract  award  and  was  refined  over  the  following  year.  The  SEMP  addressed  ihe  areas 
contained  in  the  USD  (AT&L)  Memorandum,  “Policy  for  Systems  Engineering  in  DoD,” 
February  20,  2004:  the  systems  engineering  processes  to  be  applied  to  the  program,  the 
system’s  technical  baseline  approach,  and  the  integration  of  systems  engineering  into  the 
program’s  integrated  product  teams  (IPTs).  Specifically,  the  SEMP  contains  the  program 
description,  technical  baseline  planning  and  controls,  program  planning  and  controls,  the 
IPT  structure,  the  systems  engineering  processes  to  be  used,  engineering  specialty 
integration,  and  product  transition. 

"The  SEMP  did  not  include  event-driven  technical  reviews,  with  entrance  and  success 
criteria  (IG  Report  page  8)". 

The  CMDS  Project  Office  non-concurs  with  this  statement. 
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Technical  reviews  are  specified  in  the  SEMP  and  arc  also  specified  in  the  Integrated 
Master  Schedule.  While  the  entrance  and  success  criteria  are  not  contained  in  the  SEMP 
itself,  there  ate  two  important  documents  incorporated  into  the  SEMP  by  reference:  the 
Raytheon  Integrated  Defense  Systems  (IDS)  “Systems  Engineering  Practices  and 
Procedures"  and  Raytheon’s  “Integrated  Product  Development  System  (IPDS)”.  These 
outline  the  detailed  event-driven  technical  review  process  that  was  used  on 
SLAMRAAM.  The  technical  review  process  contains  a  series  of  ten  “gates”  that  a 
program  goes  through,  spanning  program  startup  through  program  closure.  Gates  one 
through  four  are  internal  to  Raytheon  and  outline  their  proposal  generation  and  review 
process.  Gates  five  and  beyond  are  customer  reviews. 

Gate  5:  Program  Startup  Readiness  Review 

Gate  6:  System  Requirements  and  Concept  Design  Review 

Gate  7:  Preliminary  Design  Review  (PDR) 

Gate  8:  Critical  Design  Review  (CDR) 

Gate  9:  Test/Ship  Readiness  Review 

Gate  10:  Production  Readiness  Review  (functional  audits) 

Each  of  these  gates  contains  detailed  criteria  for  conducting  the  review.  These  criteria 
include  areas  to  be  assessed,  entry  and  exit  criteria,  and  review  products  required.  The 
criteria  are  crossed  referenced  to  the  Raytheon  IPDS  and  their  Capability  Maturity  Model 
Integration  (CMMI)  programs.  For  example,  a  typical  design  review  contains  over  50 
scored  criteria. 

In  addition,  the  SLAMRAAM  Program  Manager  issued  a  Design  Review  Campaign  plan 
in  August  2005  to  further  address  the  design  review  process  to  be  used  by  Raytheon 
following  completing  its  Recovery  effort  to  correct  the  initial  contract  overrun.  The  plan 
included  design  review  guidelines,  agenda  topics,  entry  and  exit  criteria,  and  a  detailed 
assessment  checklist  (in  addition  to  the  Raytheon  Gate  checklists).  This  plan  was  made 
available  to  the  IG  team. 

The  SLAMRAAM  Product  Office  also  followed  the  Program  Executive  Office  (PEO) 
Management  Control  Checklist  for  conducting  reviews,  PEO  Policy  04-36.  This  policy 
contained  important  requirements  for  conducting  event -driven  reviews.  The  PEO 
chartered  an  Independent  Assessment  Team  (IAT)  to  oversee  the  SLAMRAAM  review 
process.  The  IAT  was  chaired  by  the  PEO  Director  of  Independent  Assessments  and 
contained  subject  matter  experts  from  outside  the  SLAMRAAM  program.  This  team 
ensured  that  entry  and  exit  criteria  were  met  and  scored.  The  team  briefed  the  PEO  on  the 
review  results. 

All  reviews  were  supported  by  the  required  documentation,  action  items,  and  after  action 
reports.  The  IAT  reports  were  provided  to  the  PEO  and  the  SLAMRAAM  Product 
Manager  received  out-briefs. 
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"Technical  reviews  measure  contractor  progress  toward  attaining  well-defined  technical, 
cost,  and  schedule  goals;  assess  system  technical  maturity  and  risk;  and  provide 
information  to  support  program  decisions.  Without  tying  technical  reviews  to  event- 
driven  criteria  through  established  entrance  and  success  criteria  for  these  reviews,  the 
product  manager  cannot  ensure  that  technical  reviews  are  held  only  when  program 
accomplishments  make  them  warranted  and  meaningful  in  measuring  program  results. 
(IG  Report  page  9). " 

The  CMDS  Project  Office  concurs  with  this  statement.  That  is  why  SLAMRAAM 
employed  a  detailed,  in-depth  event  driven  review  process  based  on  industry  best 
practice,  with  senior  leadership  independent  oversight.  This  process  was  fully 
documented,  although  not  in  the  SEMP 

"The  SEMP  did  not  include  the  system 's  technical  baseline  approach.  The  technical 
baseline  approach  describes  how  the  project  manager  will  develop,  manage,  and  use  the 
technical  baseline  to  control  system  requirements,  design,  integration,  verification,  and 
validation.  Without  identifying  the  planned  technical  baseline  approach,  the  product 
manager  cannot  effectively  control  system  requirements,  design,  and  integration  from  an 
engineering  perspective  (IG  Report  page  9).  ” 

The  CMDS  Project  Office  non-concurs  with  this  statement. 

The  SEMP  outlines  the  technical  baseline  approach  in  a  summary  fashion.  In  addition  to 
the  information  contained  in  the  SEMP,  the  technical  performance  baseline  is  contained 
in  the  Government  controlled  System  Performance  Specification  and  various  contractor 
specifications.  A  detailed  specification  tree  is  maintained.  All  documentation  is  available 
on  the  SLAMRAAM  Integrated  Data  Environment  and  requirements  are  accessed 
through  requirements  database  tools  The  development  effort  is  further  controlled  by  a 
detailed  Integrated  Master  Schedule. 

“The  contractor-developed  SEMP  did  not  fully  define  all  systems  engineering  processes 
applied  to  the  SLAMRAAM.  Specifically,  the  SEMP  only  identified  the  processes  that  the 
contractor  initially  planned  to  complete  during  the  system  development  and 
denumstration  phase  of  the  acquisition  process  and  the  contractor  had  not  updated  it  to 
implement  the  results  of  the  completed  engineering  processes,  such  as  the  system 
requirements  review.  Prepared  and  validated  as  required,  the  product  office  SEP  would 
have  identified  key  systems  engineering  processes  during  the  system  development  and 
demonstration  phase,  included  updates  based  on  completed  technical  reviews,  and 
described  how  the  systems  engineering  process  would  support  the  technical  products  of 
each  acquisition  phase  (IG  Report  page  9).  ” 

The  CMDS  Project  Office  non-concurs  with  this  statement. 

As  discussed  above,  the  program  initially  had  no  requirement  nor  was  resourced  to 
produce  or  update  a  SEP  until  the  next  program  milestone.  The  contractor  developed 
SEMP,  the  Raytheon  Integrated  Defense  Systems  (IDS)  “Systems  Engineering  Practices 
and  Procedures”,  Raytheon’s  “Integrated  Product  Development  System  (IPDS),  the  joint 
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Raytheon  -  Government  Gate  review  process  and  Design  Review  Campaign  plan,  PEO 
policies,  the  Independent  Assessment  Team,  and  the  Government  controlled  technical 
baseline  taken  together  met  the  intent  of  the  SEP  guidance.  These  processes,  when  looked 
at  as  a  whole,  provided  the  product  manager  with  well-defined  systems  engineering 
process  and  technical  baseline  approach  for  the  SLAMRAAM.  In  addition,  there  were 
event-driven  technical  reviews  with  well  defined  entrance  and  success  criteria,  thus 
helping  the  product  manager  to  effectively  manage  the  systems  engineering  process. 

These  systems  engineering  processes  permitted  the  program  to  recover  from  a  major 
contractor  performance  shortfall  one  year  into  the  program  to  a  highly  successful  Critical 
Design  Review  15  months  later.  The  program  currently  has  a  firm  technical  baseline  and 
is  executing  to  cost  and  schedule.  If  funding  were  made  available  in  2006.  the  current  rc- 
baseline  effort  would  have  been  unnecessary. 

‘‘Recommendation  B:  lVe  recommend  that  the  Product  Manager,  Surface-Launched 
Advanced  Medium-Range  Air-to-Air  Missile  revise  the  draft  systems  engineering  plan  to 
establish  a  comprehensive  systems  engineering  plan  (in  accordance  with  Under 
Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  memorandums  “Policy 
for  Systems  Engineering  in  DoD,”  February  20,  2004 ;  “Policy  Addendum  for  Systems 
Engineering, "  October  22,  2004 ;  and  “ Implementuig  Systems  Engineering  Plans  in  DoD 
-  Interim  Guidance, "  March  SO,  2 004)  that  includes  planning  for:  technical  reviews  that 
are  event-driven,  with  entrance  and  success  criteria;  technical  baseline  approach  for 
developing  the  system;  and  systems  engineering  processes  applied  to  the  Surface- 
Launched  Advanced  Medium-Range  Air-lo-Air  Missile.  (IG  Report  page  10).” 

The  CMDS  Project  Office  concurs:  The  SEP  has  been  and  will  continue  to  be  prepared 
for  the  Milestone  C  decision  in  accordance  with  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  memorandums  “Policy  for  Systems  Engineering 
in  DoD,”  February  20,  2004;  “Policy  Addendum  for  Systems  Engineering,"  October  22, 
2004;  and  “Implementing  Systems  Engineering  Plans  in  DoD  -  Interim  Guidance,” 

March  30, 2004. 


5.  Finding  C  -  Establishing  Defense  Contract  Management  Agency  Support 
Responsibilities 

The  CMDS  Project  Office  non-concurs  with  tills  finding. 

"the  commander  did  not  fully  describe  DCMA  activities  to  support  the  SLAMRAAM 
program  in  a  surveillance  plan  and  establish  letters  of  delegation  (LOD)  with  other 
DCMA  contract  management  offices  for  surveillance  at  three  of  four  SIAM RA  AM 
subcontractors  (IG  Report  page  11)." 

The  CMDS  Project  Office  non-concurs  with  this  statement. 

Regardless  of  the  content  or  structure  of  the  MOA  or  existence  of  LOD's  to 
subcontractors,  the  SLAMRAAM  Product  Manager  received  and  is  receiving  excellent 
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Page  16 


Page  13 


support  from  IX’MA.  Representatives  from  various  functional  areas  within  DCMA 
participate  in  all  FPTs  and  DCMA  support  was  and  continues  to  be  focused  based  on 
current  program  need  and  phase. 

The  program  was  able  to  access  DCMA  at  Raytheon  TRS  via  the  Sentinel  program,  part 
of  the  CMDS  Project  Office,  as  needed.  An  LOD  was  established  with  DCM  A  in  Norway 
and  was  used  frequently  to  keep  the  SLAMRAAM  Product  Manager  informed. 

"Specifically,  while  a  DCMA  support  program  integrator  at  Boeing  Huntsville  stated  that 
he  provided  the  project  manager  with  regular  reports  and  analysis  of  Boeing ’s  progress 
on  the  IFCS,  he  stated  that,  without  an  approved  LOD,  his  reporting  was  inf  ormal  and 
did  not  include  earned  value  management  cost  and  schedule  analysis. ...  Therefore,  we 
believe  that  the  formalized  reporting,  to  include  earned  value  management,  that  the 
DCMA  support  program  integrator  at  Boeing  Huntsville  could  have  provided  under  an 
LOD  would  have  given  tlw  project  manager  more  meaningful  information  on  the 
subcontractors'  progress  towards  satisfying  SLAMRAAM  cost,  schedule,  and 
performance  requirements  (IG  Report  page  14), " 

The  CMDS  Project  Office  non-concurs  with  this  statement. 

DCMA  located  at  Boeing  supported  the  IPT  even  though  the  LOD  was  not  established.  A 
DCMA  Support  Program  Integrator  (SP1)  was  at  the  Boeing  Huntsville,  AL  plant.  This 
position  is  listed  in  Annex  D  of  the  MOA.  The  monthly  reports  provided  to  the  Project 
Office  reflected  status  received  from  the  DCMA  Boeing  SPI.  The  SPI  and  SLAMRAAM 
Product  Office  personnel  interfaced  on  a  near  daily  basis  since  both  were  located  in 
Huntsville.  This  daily  interface  enhanced  overall  involvement  of  DCMA  Boeing’s 
support  to  the  program.  The  Program  Integrator  provided  (he  Project  Manager  with 
meaningful  information  on  the  subcontractor's  progress  towards  satisfying  SLAMRAAM 
cost,  schedule,  and  performance  requirements. 

Boeing  Earned  Value  Management  (EVM)  data  were  available  to  the  IPT;  the 
SLAMRAAM  PM  did  not  have  to  task  DCMA  to  provide  these  data.  Boeing  data  was 
also  contained  in  the  monthly  EVM  report  delivered  per  the  contract.  The  Boeing  effort 
was  in  a  separate  Work  Breakdown  Structure  (WBS)  element  and  the  Raytheon  program 
management  team  provided  whatever  EVM  data  on  Boeing  that  the  SLAMRAAM  PM 
requested. 

In  summary,  even  without  an  LOD  in  place,  the  SLAMRAAM  PM  received  the  required 
level  of  support  from  DCMA  in  regard  to  the  Boeing  subcontract.  There  is  no  evidence  to 
suggest  that  the  lack  of  an  LOD  affected  the  quality  or  quantity  of  information 
concerning  subcontractor  progress  or  status.  There  is  no  quantifiable  evidence  to  correlate 
that  a  lack  of  the  LOD  affected  the  support  provided,  only  qualitative  opinion. 

“DCMA  did  not  have  information  needed  to  provide  the  project  manager  with  informed 
recommendations  regarding  contractor  progress  toward  attaining  contract  cost, 
schedule,  and  performance  requirements  (IG  Report  page  11)." 

The  CMDS  Project  Office  non-concurs  with  this  statement. 

10 


54 

FOR  OFFICIAL  USE  ONLY 


Final  Report 
Reference 


DCMA  was  and  is  a  member  of  all  SLAMRAAM  IPTs  and  the  SLAMRAAM  Program 
Management  Team  (PMT).  DCMA  participates  in  all  PMT  meetings.  DCMA  delivers 
monthly  reports  to  the  PM  that  covers  technical  issues,  programmatic  issues,  contract 
status,  and  Earned  Value  analyses.  DCMA  representatives  were  invited  to  many 
Raytheon  internal  reviews  and  the  after  action  reports  provided  to  the  SLAMRAAM  PM 
were  timely  and  provided  critical  insight  into  program  execution  status. 

DCMA  did  provide  timely  and  meaningful  insights  consistently  and  routinely  to  the 
SLAMRAAM  PM  and  the  Project  Manager,  CMDS.  All  requests  by  the  SLAMRAAM 
PM  to  refocus  DCMA  efforts  or  resources  to  better  reflect  program  needs  were 
accomplished  in  a  timely  manner.  Any  issues  associated  with  the  MOA,  the  surveillance 
plan,  or  LODs  did  not  stand  in  the  way  of  DCMA  providing  the  required  support,  as  and 
when  needed. 

“Without  a  focused  and  comprehensive  MOA,  supported  by  a  surveillance  plan  and 
LODs,  DCMA  staff  were  not  in  a  position  to  provide  the  Project  Manager,  CMDS  with 
timely  and  meaningful  insights  and  recommendations  regarding  the  contractor  progress 
toward  attaining  cost,  schedule,  and  performance  contract  requirements  for  SLAMRAAM 
( IG  Report  page  15).  ” 

The  CMDS  Project  Office  non-concurs  with  this  conclusion. 

There  is  no  quantifiable  evidence  to  show  that  any  issues  with  the  MOA  affected  the 
support  provided  by  DCMA  or  the  quality  or  quantity  of  information  provided 
concerning  contractor  progress  and  status.  The  quality  of  an  MOA  docs  not  necessarily 
equate  to  quality  support,  nor  can  one  conclude  that  problems  with  an  MOA 
automatically  equate  to  problems  with  the  support.  In  this  case  there  is  no  evidence  that 
such  a  conclusion  is  valid. 

"Recommendation  Cl:  We  recommend  that  the  Commander,  Defense  Contract 
Management  Agency  Space  Sensors  and  Communications  Operations.  Raytheon 
Integrated  Defense  Systems  and  the  Project  Manager,  Cruise  Missile  Defense  Systems 
revise  the  memorandum  of  agreement  for  the  Surface-Launched  Advanced  Medium- 
Range  Air-to-Air  Missile  to:  a.  Reference  the  current  Defense  Contract  Management 
Agency  Instruction  and  Defense  Contract  Management  Agency  Guidebook,  b.  Include  an 
annex  that  documents  a  cause -and -effect  analysis  linking  Defense  Contract  Management 
Agency  performance  metrics  and  standards  to  the  customer-desired  outcome,  in 
accordance  with  the  Defense  Contract  Management  Agency  Instruction  ( IG  Report  page 
15)." 

The  CMDS  Project  Office  concurs.  CMDS  and  DCMA  are  in  the  process  of  establishing 
a  new  MOA  through  the  newly  selected  DCMA  Program  Integrator  that  will  reflect 
current  2007  DCMA  guidance.  This  MOA  will  reflect  the  more  recent  changes  to  the 
Performance  Based  Management  process  that  have  been  made  since  the  previous  May 
2006  MOA  was  issued,  including  lessons  learned  through  the  maturation  of  PBM  within 
DCMA.  DCMA  Raytheon  IDS  plans  on  getting  this  new  MOA  signed  in  the  fall  of  2007. 
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The  CMDS  Project  Office  has  no  comment  on  Recommendations  C2  and  C3  since  these 
do  not  apply  to  CMDS. 

6.  Finding  D  -  Implementing  Information  Assurance  Guidance 

"The  SLAMRAAM  Product  Office  changed  to  an  information  assurance  strategy  that 
does  not  comply  with  issued  and  approved  DoD  information  assurance  requirements. 
Specifically,  the  product  office,  at  the  direction  of  the  U.S.  Army  Chief  Information 
Officer,  began  following  the  unapproved  DoD  Information  Assurance  Certification  and 
Accreditation  Process  (DIACAP)  guidance  that  was  in  coordination  within  DoD,  but  not 
approved  for  implementation.  The  DoD  Information  Technology  Security  Certification 
and  Accreditation  Process  (DI7SCAP)  remains  the  official  DoD  information  assurance 
guidance.  (IG  Report  page  16).  " 

The  CMDS  Project  Office  concurs  with  this  statement  as  written.  However,  the 
SLAMRAAM  Program  is  following  the  direction  issued  by  the  Department  of  the  Army 
and  must  continue  to  do  so  until  directed  otherwise.  The  SLAMRAAM  Product  Manager 
is  taking  all  steps  possible  to  ensure  that  the  IA  program  does  not  place  the  information 
contained  in  the  SLAMRAAM  system  at  greater  risk  of  loss,  misuse,  or  unauthorized 
access  to  or  modification  of  the  information  contained  in  the  system. 

"D.  I.  We  recommend  that  the  Army  Chief  Information  Officer  rescind  all  verbal 
or  written  direction  given  to  Army  acquisition  program  managers  to  follow  the 
interim  guidance  on  DoD  Information  Assurance  Certification  and  Accreditation 
Program  in  developing  weapon  systems. 

D.2.  We  recommend  that  the  Product  Manager,  Surface-Launched  Advanced 
Medium-Range  Air-to-Air  Missile  return  to  following  the  DoD  Information 
Technology  Security  Certification  and  Accreditation  Process  in  developing  the 
Surface-Launched  Advanced  Medium-Range  Air-to-Air  Missile,  as  documented 
in  the  information  assurance  strategy  and  in  accordance  with  DoD  Instruction 
5200.40,  "DoD  Information  Technology  Security  Certification  and  Accreditation 
Process  (DITSCAP),  "  December  30,  1997.  (IG  Report  page  19)  " 

The  CMDS  Project  office  has  no  comment  on  the  Recommendations.  Recommendation 
D1  does  not  apply  to  CMDS.  Recommendation  D2  would  require  that  the  SLAMRAAM 
Product  Manager  ignore  official  Department  of  the  Army  direction. 

2i 

APPROVED:  Edward  L.  Mullin 

COL,  AD 
Project  Manager 
CMDS  Project  Office 
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Department  of  the  Army,  Director  of  Combat 
Developments,  U.S.  Army  Air  Defense  Artillery 
School  Comments 


DEPARTMENT  OF  THE  ARMY 

HEADQUARTERS,  U.  S.  ARMY  AIR  DEFENSE  ARTILLERY  SCHOOL 
5800  CARTER  ROAD 
FORT  BLISS,  TEXAS  79916-3802 


REPLY  TO 
ATTENTION  OF: 


ATSA-CD 


MEM0RANDIR4  FOR.  Department  of  Defense  Inspector  Ccncral  Office,  400  Army  Navy 
Drxve  (Room  801)  Arlington,  VA  22202-4704 

SUBJECT:  USAADA8CH  DCD  Comments  to  DRAFT  Report  on  the  Acquisition  of 

Surface-Launched  Advanced  Medium-Range  Air-to-Air  Missile  (Project  No.  D2007- 
DOOOAE-006D . 000) ,  August  17,  2007 


1 .  References : 


a.  DRAFT  Report  on  the  Acquisition  of  Surface-Launched  Advanced  Medium- 
Range  Air-to-Air  Missile  (Project  No.  D2007-DOOOAS-OQ60 . 000) ,  August  17,  2007 


2.  Enclosure: 

USAADASCH  DCD  Response  to  DoDIG  Draft  Audit  Report  ref  (a)  contains  the 
finding  and  recommendation. 


3.  My  point  of  contacts  for  this  draft  audit  are  | 


568- 

368- 


or  by  email  at 
or  by  email  at 


|8 us. arcy.mil  and 
p us . army . mil 


at  (915) 
at  (915) 


FOR  THE  COMMANDANT: 


Enel 

as 


DIRECTOR  COMBAT  DEVELOPMENTS 


-UNCLASS  IFIED/*#*- 
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Team  Members 

The  Department  of  Defense  Office  of  the  Deputy  Inspector  General  for  Auditing, 
Acquisition  and  Contract  Management  prepared  this  report.  Personnel  of  the 
Department  of  Defense  Office  of  Inspector  General  who  contributed  to  the  report 
are  listed  below. 

Richard  B.  Jolliffe 
John  E.  Meling 
Harold  C.  James 
Andrew  D.  Greene 
Breon  E.  Dehoux 
Douglas  W.  Slaughter 
Steven  P.  Mazur 
Caryn  M.  Chambers 
Jaime  A.  Bobbio 
Charles  S.  Dekle 
Meredith  H.  Johnson 
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